I am using ab to load test my server. As I increase the concurrency number from 1, the requests per sec. goes up. I also see time per request (mean) goes up, while time per request (mean across all concurrent threads) goes down. However, once the concurrency number gets past a certain number, requests per sec. stays constant, time per request (mean) goes up linearly with concurrency number, while time per request (mean across all concurrent threads) stays constant.
How do I interpret this result? Is the latency of each request actually increasing? Why does requests per second stay constant? Does it mean the server is queuing up requests and serving them one after another at a constant rate? Is ab simply calculating time per request by looking at total time elapsed, number of concurrent threads and total number of requests?