1

I had an issue the other day and you guys were very helpful. I've been racking my brain on this one for over a week. Basically, I have several nginx servers reverse proxying a python application running on uwsgi that are serving very slow http responses when latency is (somewhat) high. Each server has a 2Gb internet connection and I'm connected at over 200Mb myself. I'm 50ms away from the server.

When I run apachebench against the server in the same datacenter, these are the results:

Document Length:        68093 bytes

Concurrency Level:      1
Time taken for tests:   0.912 seconds
Complete requests:      10
Failed requests:        0
Total transferred:      685380 bytes
HTML transferred:       680930 bytes
Requests per second:    10.96 [#/sec] (mean)
Time per request:       91.217 [ms] (mean)
Time per request:       91.217 [ms] (mean, across all concurrent requests)
Transfer rate:          733.76 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    89   91   1.9     91      95
Waiting:       81   84   1.9     83      87
Total:         89   91   1.9     91      95

Percentage of the requests served within a certain time (ms)
  50%     91
  66%     91
  75%     93
  80%     93
  90%     95
  95%     95
  98%     95
  99%     95
 100%     95 (longest request)  

Which is about what I would expect.

However, when I run apachebench from my computer, this is what I get:

Document Length:        68093 bytes

Concurrency Level:      1
Time taken for tests:   2.827 seconds
Complete requests:      10
Failed requests:        0
Total transferred:      685380 bytes
HTML transferred:       680930 bytes
Requests per second:    3.54 [#/sec] (mean)
Time per request:       282.739 [ms] (mean)
Time per request:       282.739 [ms] (mean, across all concurrent requests)
Transfer rate:          236.73 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       47   48   1.6     48      51
Processing:   223  234  15.7    230     278
Waiting:      130  138  13.9    134     177
Total:        272  283  16.7    277     328

That's about three times the processing time and about a third as fast transfer rate. The only difference is the latency. Why would latency cause such a massive drop in transfer speed? This causes noticable delays on our website. It appears that nginx doesn't send the entire payload all at once, instead waiting for ACK packets before sending more of the payload, causing the latency to decrease throughput. I looked through tcpdump and it appears that nginx is only sending 4k of data per packet as well.

Do any of you have any recommendations on how I speed up this connection so that it utilizes the full bandwidth available? Thank you!

0

When I did a load test on a website I'm developing I got 250 transactions per second on a fast connection with a fast computer, from a server in the same data center I got 2500 transactions per second.

Latency is the answer, as you said. It just adds time before work can be done, so waiting time. Load will be lower too.

You only used one connection and did ten requests (concurrency is one). If your latency is 250ms then you have 2.5s of waiting time testing from a remote connection - maybe double that as data goes both ways. If you only had 5ms latency then this is reduced to 50ms, which is imperceptible.

If you want to use your full connection to load test your system you need to run many tests in parallel - 20, 50, maybe 1000. This is increasing the concurrency. In Siege this is done as "benchmark mode" or "load test" mode - I forget the exact name.

| improve this answer | |
  • Thank you for your answer. I'm not trying to see what the maximum load the server can handle is (I know its significant). I'm trying to decrease the response/transfer time for the data since it delays rendering of the page in a browser. – George Sibble Jan 11 '16 at 22:16
  • Latency isn't something you can do much about. You can set caching headers to cache static resources (js/css/images), and some pages may be able to be cached so they're faster the second time around. You can gzip pages to reduce transmission time. – Tim Jan 11 '16 at 22:28
  • You can of course do things like use a CDN to move static content closer to your users (Cloudflare/Cloudfront/Akami/MaxCDN/etc), and you may be able to architect your application to have servers in different places. You run into problems with data replication if you're not careful. – Tim Jan 11 '16 at 22:44
  • Or leverage the multiplexing from HTTP/2 for example, so more comes at the same time. – JayMcTee Jan 12 '16 at 14:16
  • HTTP/2 (which requires TLS1.2) vs using a CDN is an interesting question regarding performance. HTTP/2 doesn't help latency directly, it just avoids connection overheads, which I guess does reduce latency slightly. A CDN with a node near the user is probably going to be better, especially if the CDN supports HTTP/2 (CloudFlare does). – Tim Jan 12 '16 at 19:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.