Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have an array of EC2 t1.micro instances behind a load balancer and each node can manage ~100 concurrent users before it starts to get wonky.

i would THINK if i have 2 such instances it would allow my network to manage 200 concurrent users... apparently not. When i really slam the server (blitz.io) with a full 275 concurrents, it behaves the same as if there is just one node. it goes from 400ms response time to 1.6 seconds (which for a single t1.micro is expected, but not 6).

So the question is, am i simply not doing something right or is ELB effectively worthless? Anyone have some wisdom on this?

AB logs:
Loadbalancer (3x m1.medium)
Document Path:          /ping/index.html
Document Length:        185 bytes

Concurrency Level:      100
Time taken for tests:   11.668 seconds
Complete requests:      50000
Failed requests:        0
Write errors:           0
Non-2xx responses:      50001
Total transferred:      19850397 bytes
HTML transferred:       9250185 bytes
Requests per second:    4285.10 [#/sec] (mean)
Time per request:       23.337 [ms] (mean)
Time per request:       0.233 [ms] (mean, across all concurrent requests)
Transfer rate:          1661.35 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        1    2   4.3      2      63
Processing:     2   21  15.1     19     302
Waiting:        2   21  15.0     19     261
Total:          3   23  15.7     21     304

Single instance (1x m1.medium direct connection)

Document Path:          /ping/index.html
Document Length:        185 bytes

Concurrency Level:      100
Time taken for tests:   9.597 seconds
Complete requests:      50000
Failed requests:        0
Write errors:           0
Non-2xx responses:      50001
Total transferred:      19850397 bytes
HTML transferred:       9250185 bytes
Requests per second:    5210.19 [#/sec] (mean)
Time per request:       19.193 [ms] (mean)
Time per request:       0.192 [ms] (mean, across all concurrent requests)
Transfer rate:          2020.01 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        1    9 128.9      3    3010
Processing:     1   10   8.7      9     141
Waiting:        1    9   8.7      8     140
Total:          2   19 129.0     12    3020
share|improve this question
3  
"Worthless" is a bit strong, don't you think? Plenty of firms run sites with hundreds or thousands of instances fronted by ELB. –  khoxsey Aug 20 '12 at 19:41
    
i know! thats why i'm saying, I'm clearly doing something wrong here. I cant figure out why the numbers aren't supporting the concept. I have 6 instances behind a load balancer... but the numbers show the same data as 1 instance alone. –  Jones Aug 20 '12 at 20:01
2  
Yes, you are not reading the documentation. Micros are not intended for significant load, Amazon is quite clear about this. You are consuming your "fair use" budget and getting clamped. Your 6 micros are costing you 12c/hr, replace them with a c1.medium at 16.5/hr and retest. –  khoxsey Aug 20 '12 at 20:11
1  
Understood. An easy way to verify that you are having ELB problems is to replace the ELB with a suitable instance (let's say m1.small for now) and start up Apache in reverse-proxy mode. If you can get the desired throughput from your micro array, then it is clearly ELB and I would be very interested to see the ab output posted here, it would be incredibly helpful. –  khoxsey Aug 20 '12 at 20:37
1  
What does a longer test than 10 seconds show (i.e. several minutes)? ELB takes a while to ramp up depending on traffic history. –  ceejayoz Aug 20 '12 at 21:09

4 Answers 4

Micro instances are not designed for sustained load. They allow bursting CPU, but after a short period of time (think 15-30 seconds) at heavy load they will be severely capped.

Try it with a small instance, at least, if you want any sort of useful benchmark.

share|improve this answer
    
+1 for identifying the oxymoron: load-balanced micros are the jumbo shrimp of the cloud world. For cheap capability, switch to load-balanced c1.mediums (5 ECU per). –  khoxsey Aug 20 '12 at 19:39
    
My understanding is that an array of 6 t1.micro instances SHOULD handle 6 times the load a single instance can handle, but this isn't happening in practice - isn't that what's supposed to happen? –  Jones Aug 20 '12 at 19:57
    
No, they will not necessarily handle six times the load. Try it with a set of small/medium instances and see. –  ceejayoz Aug 20 '12 at 20:03
    
Like already said, at 250 reqs/second it's rather the ELB then the instance, thus upgrading the instance from micro to small will likely not help although can be tried, but I have serious doubts again. –  Logic Wreck Aug 21 '12 at 9:44
4  
The problem is that the load balancers need time to scale up. Not every load balancer is equal. If you go from 0 to 250 requests / second in 0 seconds then the load balancer will not be configured to handle that load. Usually takes 5-10 minutes for the load balancer to complete the upgrade. However, like ceejayoz said, the micro instances are horrible you're not going to get very far stress testing micro instances. I can easily handle 3000+ request/second with 1 load balancer without any special configuration. –  bwight Aug 21 '12 at 16:16

Make sure you haven't accidentally selected sticky load balancing. This would cause the same user to be directed to the same instance.

Micro instances weren't designed to sustain heavy load. They are for CPU bursting. I can assure you though that micro instances work fine with elastic load balances.

Don't forget there may be other ways to increase the traffic your website can cope with. Eg. Varnish

share|improve this answer

Check the load on the single servers. ELB does not balance equally the traffic to all instances when this is coming from a single IP (like in the AB test case): it just switches from one instance to another. The final load then cannot be the double of a single instance, but on average it is in any case better than directing all the traffic to one single instance (due to reduce load and faster response)

share|improve this answer
    
I believe blitz.io is supposed to use many IPs. –  ceejayoz Dec 17 '12 at 16:54
    
This answer should be the winner I think. If you ab test from a single ip address/range ELB will stick the connections to single EC2 instance regardles whether you're using sticky-sessions OR NOT. I would like to know how to workaround this. –  WooDzu Mar 25 at 23:45

The issue here is the ELB itself. Some time ago we did some performance testing and found out that the ELB starts having big latency when there are more then 250 requests/second. We found this out when testing the ELB, then doing these tests against one of the instances behind the ELB - the instance(a m1.large instance type) was doing fine(although had some load) with 250 requests/second, while the ELB with a few instances behind it was dying. At the same time, when testing the ELB the instances had little load.

My suggestion is to get an instance which will act as a load ballancer(configure nginx on it preferably for this) for the other instances and don't use any ELB.

share|improve this answer
3  
Were you going straight to 250 request/second? ELBs take a little while to scale up. Given that Netflix uses ELBs, it's pretty clear that they can handle a lot of traffic if used properly. –  ceejayoz Aug 20 '12 at 20:03
    
Yea thats basically what i'm seeing too. Not sure why this is. I know that netflix uses AWS and so it can't be a joke service, but it doesn't seem to be working here. –  Jones Aug 20 '12 at 20:36
    
Actually I was going up from like 20 requests/second, did many tests until got to 250 requests/second and got this high latency. So in general you're very wrong here. Also this was being done with m1.large instances, not t1.micro in my case. Please read carefully next time. –  Logic Wreck Aug 20 '12 at 20:42
3  
@LogicWreck I asked "were you?", not "you are!", nor did I in any way imply that you were using micros in your situation. Two touchy folks on one thread today, it would seem. Troubleshooting requires eliminating possibilities by asking questions - there's no indication in your answer of what instance type or whether you were using a ramp-up, so I asked. –  ceejayoz Aug 20 '12 at 22:18
1  
While you're instructing me to read carefully, please read my posts on your answer and tell me where I said you were using a micro instance? –  ceejayoz Aug 21 '12 at 13:35

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.