I've got a site hosted on Amazon using a EC2 instance backed by an EBS volume. On the weekends, traffic spikes and I make the instance larger which helps quite a bit -- I'm no longer seeing CPU usage spiking to 100% and the server becoming non-responsive.

However, I do notice that disk reads are also very high (can't be helped, I don't think) and I'm wondering, at what point am I going to see some sort of failure because the disk can't keep up?

As you can see from the attached screenshot, it maxed out at 80MB/minute over the weekend. Does anyone have experience with AWS and know at what point I'll have to move to multiple-load balanced instances because EBS becomes the bottleneck?

EBS Usage Graph

  • 2
    I think we need some application-layer details here. What is the disk read activity? Random-access for images/assets? Large sequential reads (e.g. database backups or analytics)? Note that you can run multiple EBS volumes attached to a single instance doing software RAID-0 to improve things. – rmalayter Oct 3 '11 at 15:26
  • most of those reads are smaller files (less then 1MB, many less then 500k) but not "tiny" files. I have no idea if this is a load for a EBS volume or if this amount of traffic is trival -- and have no idea how to figure it out. – ESW Oct 3 '11 at 15:30
  • I was just writing out a reply featuring RAID and EBS when I read your comment, malayter. I'd add that RAID10 might be a more flexible solution, as it should improve reads AND writes. – tsykoduk Oct 3 '11 at 21:55
  • @tsykoduk In general, all the AWS storage types have somewhat non-obvious performance and even though it might seem that RAID10 could improve performance it is really better to try benchmarking with the kind of access patterns and load that your real application is likely to experience. Even then your performance is likely to vary wildly at times in the AWS environment compared to a self-hosted solution where you have more control-- and thus more predictability-- over the storage performance. In general, have more than one volume in a RAID configuration is likely to have a positive impact. – aculich Feb 16 '12 at 8:55

The first thing to keep in mind that will have the most impact on your I/O performance is the instance type that you're using.

Instance Type   I/O Performance
-------------   ---------------
t1.micro        Low
m1.small        Moderate
m2.xlarge       Moderate
c1.medium       Moderate
m1.large        High
m1.xlarge       High
m2.2xlarge      High
m2.4xlarge      High
c1.xlarge       High
cc1.4xlarge     Very High (10 Gigabit Ethernet)
cc2.8xlarge     Very High (10 Gigabit Ethernet)
cg1.4xlarge     Very High (10 Gigabit Ethernet)

As for EBS volumes and the performance that you'll get, as the AWS FAQ suggests, you'll need to benchmark your application to see what to expect:

Q: What kind of latency and throughput rates can I expect to see from Amazon EBS volumes? The latency from an Amazon EC2 instance to an Amazon EBS volume is similar to the latency you would see from the local Amazon EC2 instance storage drive. I/O rates can vary significantly based on the size of the requests, the randomness of the access patterns, and the caching strategy used by the application. As such, the most accurate measure is to benchmark your specific application on an Amazon EBS volume.

What this means is that the EBS rates you get many not necessarily be worse or better than local instance storage; it really depends on your data access behavior.

Further info is on the AWS EBS page:

Amazon EBS Volume Performance

Amazon EBS volumes are designed to offer higher throughput than Amazon EC2 instance stores for applications performing a lot of random accesses across your data set. You can also attach multiple volumes to an instance and stripe across the volumes to achieve further increases in throughput.

The exact performance will depend on the application (e.g. random vs. sequential I/O or large vs. small request sizes), so the best measure is to benchmark your real applications against the volume. Because Amazon EBS volumes require network access, you will see faster and more consistent throughput performance with larger instances.

Also keep in mind that the I/O performance not only includes the disk IO, but also the network traffic... so, the more network traffic your instance gets the less disk IO you'll get.

Depending on what you're serving, in-memory caching of objects may help considerably if that is possible for your type of application.

Also, here are some blog posts that benchmark the performance of EBS and local (ephemeral) volumes in various RAID configurations and tweaks for getting good IO performance:

EC2 Ephemeral Disks vs EBS Volumes in RAID

Amazon EC2 I/O Performance: Local Ephemeral Disks vs. RAID 0 Striped EBS Volumes

Getting Good IO from Amazon's EBS

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.