1

I'm running a simple httperf stress test on a quad-core Nginx server, pointed at an HTML/PHP document. Large connection rates quickly begin experiencing high connection and response times (see the median connection time and the reply time in the results below). What makes this puzzling is when the test is repeated on an Apache webserver serving identical content.

The Apache server doesn't break a sweat relatively. The only difference I notice is in the "Net I/O" values, which are much larger when testing the Nginx server (3315.6 KB/s vs 55.5 KB/s). The response time also has a large contribution from "transfer," (849.6 ms) while the Apache server has "0.0" there. My first thought was that web caching was not working on the Nginx server, causing more data to be transferred, but this was not the case, and httperf isn't a browser anyway.

My Nginx configuration should theoretically be able to handle this stress test just fine. I suspect the data transfer volumes are the reason for the poor performance.

So my question is: what about an Nginx configuration could explain this difference in data transfer / content-length relative to the Apache server hosting identical content?

Here are the httperf results on both servers for a simple 10-second test with 1000 connections:

NGINX

httperf --hog --server xxx.xxx.xxx.xxx --uri /test.html --num-conns 1000 --rate 100
httperf --hog --client=0/1 --server=xxx.xxx.xxx.xxx --port=80 --uri=/test.html --rate=100 --send-buffer=4096 --recv-buffer=16384 --num-conns=1000 --num-calls=1
httperf: warning: open file limit > FD_SETSIZE; limiting max. # of open files to FD_SETSIZE
Maximum connect burst length: 1

Total: connections 1000 requests 1000 replies 1000 test-duration 11.776 s

Connection rate: 84.9 conn/s (11.8 ms/conn, <=214 concurrent connections)
Connection time [ms]: min 158.2 avg 1608.1 max 2695.7 median 1729.5 stddev 532.2
Connection time [ms]: connect 373.9
Connection length [replies/conn]: 1.000

Request rate: 84.9 req/s (11.8 ms/req)
Request size [B]: 84.0

Reply rate [replies/s]: min 69.2 avg 79.0 max 88.8 stddev 13.9 (2 samples)
Reply time [ms]: response 384.6 transfer 849.6
Reply size [B]: header 194.0 content 39702.0 footer 2.0 (total 39898.0)
Reply status: 1xx=0 2xx=1000 3xx=0 4xx=0 5xx=0

CPU time [s]: user 0.18 system 11.57 (user 1.5% system 98.3% total 99.8%)
Net I/O: 3315.6 KB/s (27.2*10^6 bps)

Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0

APACHE

httperf --hog --server xxx.xxx.xxx.xxx --uri /test.html --num-conns 1000 --rate 100
httperf --hog --client=0/1 --server=xxx.xxx.xxx.xxx --port=80 --uri=test.html --rate=100 --send-buffer=4096 --recv-buffer=16384 --num-conns=1000 --num-calls=1
httperf: warning: open file limit > FD_SETSIZE; limiting max. # of open files to FD_SETSIZE
Maximum connect burst length: 1

Total: connections 1000 requests 1000 replies 1000 test-duration 10.101 s

Connection rate: 99.0 conn/s (10.1 ms/conn, <=29 concurrent connections)
Connection time [ms]: min 53.0 avg 117.7 max 3074.8 median 72.5 stddev 264.3
Connection time [ms]: connect 79.7
Connection length [replies/conn]: 1.000

Request rate: 99.0 req/s (10.1 ms/req)
Request size [B]: 88.0

Reply rate [replies/s]: min 97.0 avg 99.2 max 101.4 stddev 3.1 (2 samples)
Reply time [ms]: response 38.1 transfer 0.0
Reply size [B]: header 231.0 content 255.0 footer 0.0 (total 486.0)
Reply status: 1xx=0 2xx=0 3xx=1000 4xx=0 5xx=0

CPU time [s]: user 1.23 system 8.86 (user 12.1% system 87.7% total 99.8%)
Net I/O: 55.5 KB/s (0.5*10^6 bps)

Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0
1

It turns out the difference in response length is a methodological one. I didn't notice the Apache server was returning all 301 codes during this test, because the URLs were being re-written. I had to change the server URL and path to exactly match what the re-write rules enforce. After that, the content lengths matched up perfectly, and the Apache server actually struggled a bit more than Nginx in this test.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.