Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'd like to simulate 1000 concurrent downloads of a single file from Cloudfront. I figured I'd setup ~10-20 xlarge EC2 instances for this. Is there an obvious way I'm missing to trigger this at the same time and get the average download time while ensuring the instances aren't the bottle-neck. We REALLY need to know how much outbound bandwidth we can sustain from Cloudfront.

Thanks!

share|improve this question
    
Some auto-scaling services from AWS take time to scale up to demand. You may want to start your test at a slower pace and gradually scale up over time. If you have a specific event where you know you'll be getting a sudden flood of traffic, work with your Amazon rep so that they can pre-scale your resources in advance. –  Eric Hammond Oct 17 '12 at 21:58
    
No worries but thanks for the tip. We're good buddies over there now. We're planning 2.5 million viewer events with them. –  Publiccert Oct 17 '12 at 22:17

2 Answers 2

up vote 2 down vote accepted

I see two options.

  1. Use a messaging protocol, like AMQP to coordinate the slaves
  2. (Simpler, recommended.) Make sure all of you slaves have synchronized clocks and just set them to start the download jobs at the same time, using the "at" command or something similar. Just be sure to set up the job a few minutes before it should start, in order to have all of the slaves be ready by the start time.
share|improve this answer
    
I was thinking about doing something like that. Setup a single EC2 server. Create an image of it with my script setup with CRON. Confirm the time is accurate. Then hope for the best. Any simple way of getting accurate average throughput rates on each download? –  Publiccert Oct 17 '12 at 18:04
    
It depends on the tool used for downloading. curl or wget can give individual download rates. ab (apachebench) gives a summary of the download speeds. If you want a summary by machine, then sar or ifconfig can give you the number of packets received on the interface, but the best numbers will come from the program that does the downloading. –  edgester Oct 17 '12 at 18:20

Take a look at Soasta's products. They have various (include free) performance testing tools that produce very detailed results. It can help you pinpoint where, if any, performance bottlenecks are. So you could realistically launch 1000 concurrent downloads from a number of instances(even in various geo locations).

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.