How to interpret ab result?

Question

I am using ab to load test my server. As I increase the concurrency number from 1, the requests per sec. goes up. I also see time per request (mean) goes up, while time per request (mean across all concurrent threads) goes down. However, once the concurrency number gets past a certain number, requests per sec. stays constant, time per request (mean) goes up linearly with concurrency number, while time per request (mean across all concurrent threads) stays constant.

How do I interpret this result? Is the latency of each request actually increasing? Why does requests per second stay constant? Does it mean the server is queuing up requests and serving them one after another at a constant rate? Is ab simply calculating time per request by looking at total time elapsed, number of concurrent threads and total number of requests?

Answer 1

It sounds like your server has hit the maximum requests per second and time per request (inverses of each other, really).

Say the numbers are 100 requests per second maximum.

If your concurrency is 200 requests, what that means is that after (say) 4 seconds, you will have 200 requests made. The server will only respond to 100 requests every second, and after every response, the client makes a new request (to bring the in-flight total back to 200). That means that each request will take 2 seconds (roughly / on average) to get a response because the server essentially gets a backlog of 100 requests behind the 100 it can actually work on.

If you increase the concurrency to 400, you don't change the requests per second that the server can handle, but you do change the size of the backlog (to 300), which changes the time required for each given request (4 seconds). You still get 100 responses every second; they are just for requests 4 seconds old.

If you keep going up, at some point you'll get to the point where the server can't maintain a backlog any longer. At that point you'll probably start seeing some failed requests. The tool you have may not be able to hit that though.