So, i'm currently writing my bachelor thesis and my job is to cloudify a webservice which recommends a soundtrack for images. The main part of the recommendation process is a lookup on a ~40gb index file. The index file is read only and reading it should be as fast as possible. I also want to automatically launch additional instances based on demand. I did some research and have 3 possible ways to do it

  1. Increase the root partition of the ec2 instance (to ~50gb), store the index file on the root partition and create an AMI. The benefit of this approach is, that it is very easy to start a new instance since everything is included in the AMI. But i also read that the speed of the root partition is very slow.
  2. Store the data on a EBS volume, create a snapshot of it and whenever i launch a new instance, i create a new EBS Volume from the snapshot an attach it to the instance. I suppose this is the best way to do it but launching a new instance is a tad more complex
  3. Storing the index file on S3 and whenever a new ec2 instance is launched, i download the file to the ephemeral storage of the instance. Problem with that approach is that it takes more time until the new instance is operable as well as the the cost for traffic.

I was also thinking about doing a Raid0 setup to improve the read performance on the index file. So which approach should i use?

  • In point 3 you mention the cost for traffic. I thought that transfers between S3 and EC2 instances were free as long as they were located in the same datacentre – Matthew Steeples Oct 4 '11 at 12:01
  • it's free in US, but unfortunately not in EU – kev Oct 4 '11 at 12:24
  • It's free as long as you're within the same datacentre. See aws.amazon.com/s3/#pricing: "There is no Data Transfer charge for data transferred between Amazon EC2 and Amazon S3 within the same Region" – Matthew Steeples Oct 4 '11 at 13:31
  • Then the book i read is either outdated or erroneous because it states that "There are no charges for data transferred between Amazon's EC2 environment and S3 buckets located in the United States. Data transferred between EC2 and S3 buckets located in Europe incur the standard data-transfer charges." Good to know anyway – kev Oct 5 '11 at 18:53
  • Possibly either. If you hosted your EC2 in the states and S3 in Europe then you would incur charges, but if they're both in the same region it's free – Matthew Steeples Oct 5 '11 at 19:14

Is this webservice already existing or something you're making or...? Because my answer would be to not cloudify it at all. I'd get a decent server with SSD's in it for maximum read performance.

But you'd also have to figure out what you're getting from the "cloud" side. "Clouds" just allow you to create many instances in a hopefully geographically diverse set of data centers; even then you don't guarantee that your end users will get good response time (like with Amazon, how do you get consistent performance if someone hits your server in a datacenter in New York and your client is in Australia?)

Where are your performance bottlenecks and how do you decouple elements to increase performance? Disk reads themselves would be taken care of with SSD's. "Clouds" don't magically increase performance; it's very much a function of architecture of the application. I haven't tested it and would like to know general numbers if someone has, but the proposal of launching various instances on demand when you are looking for increased performance seems like it would incur a greater overhead that would kill your database performance.

Also, you're focusing on disk when you might want to take a look at caching the @#% out of it. No matter how fast your disk performance you're not going to outstrip a good set of caching servers to keep the records hot in memory versus cold on disk. Again, a function of architecture of the application. And that's another thing that can hurt you in launching more VM's; spinning up VM's can kill caches and cause a lag before the caches are "primed", so to speak.

| improve this answer | |

Comment numbers correspond to your options above.

  1. I always recommend putting data on a separate EBS volume from the root, but if it's read only and easy to obtain when you are building AMIs, then I might make an exception. I don't see why an EBS root volume should be any slower than any other EBS volume.

  2. This is easy if you set up your AMI correctly. See options for setting up block devices in ec2-register. Amazon can automatically create volumes for you based on snapshots and attatch them to the new instance, just like they do for the root volume.

  3. There is no charge for traffic between EC2 instances and S3 in the endpoint related to the region. There are charges for EBS volumes and IO on EBS volumes.

Just because an EBS volume created from a snapshot is ready to use before a full download completes of a big file from S3, doesn't necessarily mean that EBS is faster. The volume is ready to accept operations almost immediately, but you will experience high iowait while the blocks populate from the snapshot.

Depending on the performance requirements of your application, you might find you need to "warm up" the EBS volume before you can put it into production. This is effectively the same thing as downloading it from S3. (I'd love to see some performance tests of these options.)

This all assumes that getting the index on disk is the right thing to do. For the best performance you might try loading it all into memory if you can afford instances big enough. On the other hand, simply using a high memory instance with the index on disk might eventually load it into memory anyway using standard disk buffering in the OS.

Depending on what you're indexing and how you access it, you might also take a look at SimpleDB, RDS, ElastiCache.

| improve this answer | |
  • great answer, thanks. But there actually is a traffic charge between EC2 and S3 in EU :/ – kev Oct 4 '11 at 12:20
  • 1
    If you are using EC2 region "eu-west-1", then you should be using S3 location "EU" with no data transfer charges between EC2 and S3. – Eric Hammond Oct 4 '11 at 23:42

If speed is a concern than I suggest using an InstanceStore rather than EBS.


| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.