What is the best-practice way to create EC2 instances with EBS volumes that will soon need to be very large?

Question

When launching EC2 instances, it seems most community AMI's come with a 8gb EBS volume attached as the root drive. We will definitely need larger than 8GB as our database size grows. What is the advisable way to design our system? The options I see are:

1. Use the 8GB storage until we need to expand, and then follow one of several lengthy online tutorials to expand the size of the root drive.

2. Attach a 2nd EBS volume to the instance that is larger (e.g. 60 GB) and store all data on this volume. If I want to use MongoDB or MySQL as a database, will it be easy to install the database application files on the root volume but store the data on another volume?

What is the best-practice solution?

Answer 1

Unless you're using an EBS-backed EC2 instance the AMI image is extracted to a 10GB drive image that is recreated when the instance is started. I use instance-stored EC2 instances instead of EBS-backed ones for all my server instances. I then simply make the EBS volumes whatever side I need and mount them as secondary drives.

With the EBS volumes I found that using the xfs filesystem and simply using the entire EBS volume without any partitioning was the best course of action. To increase the volume when it was necessary I would perform a snapshot of the existing EBS volume and then create a new larger volume built from the snapshot. You then simply detach the existing EBS volume and attach the new one. Once the new volume is mounted it will show as the current volume size until you run the xfs expand utility command that has to be ran while the filesystem is live. Checking the capacity after that is done will show the new larger size.

Now if you're using an Ubuntu AMI you can install the ebsmount package and create a hidden directory on the EBS volume and configure the system to actually automount using udev when the EBS volume is attached to the EC2 instance.

Answer 2

I think there are actually 3 options:

1. It is quite easy to expand an EBS volume - snapshot, create EBS volume of desired size, detach original, attach new, resize filesystem, it certainly can be done on a root volume, but I would suggest that it is far better to go with option 2.
2. 2nd EBS volume - this is my personal preference (I can't vouch for it being a 'best-practice'). I keep databases on one volume (and mount it to /var/data) and web-files on another (and mount to /var/www/html) and of course have my root volume. The advantage is that a) you can resize and snapshot each drive independently, b) you have somewhat independent reads/writes (still constrained by the network), c) even if, hypothethically, the root volume is damaged (i.e. bad software upgrade, etc) the data is easily moved to a new instance (even with snapshots, the same isn't true if everything was on the root volume).
3. A cluster file system (e.g. gluster) that allows you to distribute your files across multiple volumes and add more volumes as needed. Essentially, you would choose a size (say 10GB) and as you are running out of space, add another 10GB volume, etc. Gluster will distribute the files across your volumes - this would work for both a single instance (i.e. multiple volumes attached to one instance) or across multiple instances (i.e. multiple volumes shared across multiple instances). There is a degree of complexity (and performance overhead) though in this setup, although it does theoretically offer the best versatility.

(There is a 4th option, but it really isn't practical for something like a database - you can mount S3 as a fuse filesystem - and since you don't have to pre-allocate space on S3, you would have storage that grows with your needs. It is however, quite slow compared to EBS or ephemeral storage, and its reliability is questionable at best.)

Answer 3

It's generally best practice to have your boot drive and your data drive be different file systems, anyway, so that, for example, an overzealous log file that fills the boot drive can't bind up your client's ability to upload content(or vice versa). I've never installed Mondo, but with MySQL, it's pretty easy to designate where you want to store the data. It's even easy to move the data later down the line with an rsync, a config change, and just a couple minutes of downtime if you need to expand the drives again.

If you are going the mysql route, It should probably be mentioned that Amazon's Relational Database Service does a pretty good job of creating a stable scalable database without all of the headaches of managing your own.

Answer 4

Put your important data on (at least one) external EBS volume. For anything that you actually need performance for, use Linux MD RAID-10 across a lot of EBS volumes.