4

I need a way to reproduce the sudden burst of traffic our site experienced a while ago, to have a chance to keep our stack working.

Our load balancer reported at a certain point that some 350k requests were performed per minute, trashing everything, from the operating system to knocking off the backends. I tried looking into services that performs this kind of burst test, but it appears that when tweaking the test settings to match the desired conditions it would simply cost too much.

How can I load test my stack in a way that matches these conditions?

Improve this question
3
  • Fire up a bunch of EC2 spot instances - you can get decently-provisioned boxes for pennies.
    – EEAA
    Jan 1, 2016 at 22:23
  • What about using a few desktops with gigabit NICs connected directly to your network ? Or renting a few extra machines from your hosting provider within the same datacenter? That way you'll avoid paying for precious Internet bandwidth as you keep all your testing traffic within your own network.
    – André Borie
    Jan 1, 2016 at 22:49
  • Apache ab can certainly generate a large burst of traffic.
    – Brian
    Jan 2, 2016 at 2:49

1 Answer 1

Reset to default
8

So, this is 5,833 requests per second, which is a lot, but its doable. I would recommend using a tool called siege. Experiment with different concurrency options until you reach a peak Transactions per second: rate in siege's output. You can also make use of ab (ApacheBench) from the Apache Project, but siege is much more powerful.

Then, add more machines until you can get to a total of, say, 6,000. You can probably make cost-effective use of Amazon EC2, or even just set up a few machines on different internet uplinks in different locations with ssh access to them. You could make use of cron or at to start siege at the same time on all of the machines (ensure that the clocks are in sync first).

As far as making use of an external company to do the test, yes, this might be prohibitively expensive, and I don't think I'm allowed to point to any particular company, but I've worked with a few in the past. This is definitely something you can do on your own with a few hours of EC2 time.

Edit

As @HBruijn awesomely pointed out, there is a great open-source tool called Bees with Machine Guns which will automate the creation, teardown, and coordination of load generation via EC2. This sounds like the right approach for your situation.

Improve this answer
2
  • 1
    No worries; just remember, "i before e, except after c" (except when it's not :).
    – MadHatter
    Jan 2, 2016 at 9:38
  • 3
    An example to do this economically and automated is bees with machine guns
    – HBruijn
    Jan 2, 2016 at 9:41

Not the answer you're looking for? Browse other questions tagged .