1

I am using AWS EC2 Linux m3.xlarge to run a program called "Smart Information Retrieval System" which performs many I/O operations to/from the disk.

I have tried running the same program on my laptop and on a different cluster, and the performance is 10 times better there than in AWS EC2, even if the instance specs are much much better.

The only explanation that could explain that behaviour is if AWS was providing a single file system to store all your data and index files, but internally the blocks from the files are stored in different secondary disks scattered across the cluster.

Does it work that way in AWS EC2?

  • 1
    What are you using for the data storage? Instance-store volumes or EBS? If EBS, which type: GP2, Magnetic, or PIOPS? – Chad Smith Apr 24 '15 at 18:30
  • It's GP2 200 Gb – Arturo Apr 24 '15 at 22:08
  • Depends on the size. gp2 gives you 3 iops per Gb, so you have about 600 iops sustained throughput on that volume. Check the cloudwatch metrics for the volume and post some data here, maybe you are overrunning the iops capacity. GP2 allows you to burst up to 3000 iops, but not consistently. – Chad Smith Apr 24 '15 at 22:12
  • What instance size are you using? This also impacts the network bandwidth to the EBS storage. – Chad Smith Apr 24 '15 at 22:44
1

AWS has a variety of different storage options. Off the top of my head these include:

  • Local magnetic disks (ephemeral storage)
  • Local SSD's
  • EBS (essentially SAN/NAS)
  • EBS with Provisioned IOPS

AWS has a lot of different options for you, including the ability to pay them more for better IO performance.

You can also get high IO performance using SSD's without paying extra for PIOPS.

In summary, you probably need to educate yourself on the options available to you when you instantiate an instance. The availability of all these options is one of the things that differentiates AWS from its many competitors at the current time.

| improve this answer | |
  • It should also be noted that non-provisioned EBS can be either magnetic or SSD, it's up to you. Provisioned is always SSD. – ceejayoz Apr 24 '15 at 18:58
  • Thanks. It is GP2, is that among the best I/O? – Arturo Apr 24 '15 at 22:09
  • 1
    And, of course, you need to pre-warm new volumes before you get meaningful benchmarks. Otherwise they'll be faster, later, than they are when first deployed. – Michael - sqlbot Apr 24 '15 at 22:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.