1

I am using AWS EC2 Linux m3.xlarge to run a program called "Smart Information Retrieval System" which performs many I/O operations to/from the disk.

I have tried running the same program on my laptop and on a different cluster, and the performance is 10 times better there than in AWS EC2, even if the instance specs are much much better.

The only explanation that could explain that behaviour is if AWS was providing a single file system to store all your data and index files, but internally the blocks from the files are stored in different secondary disks scattered across the cluster.

Does it work that way in AWS EC2?

Improve this question
4
  • 1
    What are you using for the data storage? Instance-store volumes or EBS? If EBS, which type: GP2, Magnetic, or PIOPS?
    – Chad Smith
    Apr 24, 2015 at 18:30
  • It's GP2 200 Gb
    – Arturo
    Apr 24, 2015 at 22:08
  • Depends on the size. gp2 gives you 3 iops per Gb, so you have about 600 iops sustained throughput on that volume. Check the cloudwatch metrics for the volume and post some data here, maybe you are overrunning the iops capacity. GP2 allows you to burst up to 3000 iops, but not consistently.
    – Chad Smith
    Apr 24, 2015 at 22:12
  • What instance size are you using? This also impacts the network bandwidth to the EBS storage.
    – Chad Smith
    Apr 24, 2015 at 22:44

1 Answer 1

Reset to default
1

AWS has a variety of different storage options. Off the top of my head these include:

  • Local magnetic disks (ephemeral storage)
  • Local SSD's
  • EBS (essentially SAN/NAS)
  • EBS with Provisioned IOPS

AWS has a lot of different options for you, including the ability to pay them more for better IO performance.

You can also get high IO performance using SSD's without paying extra for PIOPS.

In summary, you probably need to educate yourself on the options available to you when you instantiate an instance. The availability of all these options is one of the things that differentiates AWS from its many competitors at the current time.

Improve this answer
3
  • It should also be noted that non-provisioned EBS can be either magnetic or SSD, it's up to you. Provisioned is always SSD.
    – ceejayoz
    Apr 24, 2015 at 18:58
  • Thanks. It is GP2, is that among the best I/O?
    – Arturo
    Apr 24, 2015 at 22:09
  • 1
    And, of course, you need to pre-warm new volumes before you get meaningful benchmarks. Otherwise they'll be faster, later, than they are when first deployed.
    – Michael - sqlbot
    Apr 24, 2015 at 22:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .