Here is an article about not all EC2 instances of the same type having equal computing performance: http://www.theregister.co.uk/2012/10/22/ec2_hardware_heterogeneity/

I recently ran into this issue while running some benchmarks on a handful of EC2 test servers. Some of them performed measurably worse than others, even though the code and load were the same. More interestingly, I'm seeing variation between machines that have the same /proc/cpuinfo, so I don't think I can use that to check for instance quality reliably.

I'm looking for advice on how to test for instance quality / speed / health immediately after creation. Ideally, I'd like to run a 30-second script after instance creation, and then use the results to decide whether to terminate it and try again, or to leave it up and running.

Would something like computing pi to the n-th digit x-times work? Are there tools out there that are available that might work better?

Thank you!

  • Are you testing only the CPU performance? – Jeff Ferland Feb 21 '13 at 21:51
  • Yes, CPU performance only. Although, now that you mention it, how can I know that other stuff like network performance is the same across instances? – Mike Cialowicz Feb 21 '13 at 21:53

What you want to do is create a piece of code that runs in constant computational time in a loop. Every second, the code should interrupt itself every second and print out the number of operations it has completed. This code should be multithreaded and allow you an option to pass the number of CPU cores as an option.

I think the simplest answer is to count how many AES packets you can encrypt starting with a random string and a random key for each process, or how many times you can hash a string. That should keep branch prediction and processor cache out of the equation.

As for testing memory performance, I wouldn't bother -- the CPU consumption is usually the arbiter of speed there.

As for testing network performance, this is challenging because you can't easily isolate it. You'd need multiple instances to test against to ensure that your baseline system wasn't changing. Even then, routes to other systems may have different performance characteristics.

As for disk performance, this would only be useful if you weren't using EBS-backed disks or you were testing the overall performance of EBS since its variances are independent of the instances.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.