1

I'm running Apache 2.4.17 (prefork) on a Centos 7 server. The server specs are pretty good: 2 x Xeon E5-2697 v3 @ 2.60GHz, 512GB of RAM and 8 SSDs in RAID 10.

This Apache server is used to run a PHP software.

The normal load is between 8 and 15 but due to the server specs, it's very responsive. Also, the number of requests/sec is usually below 300.

I noticed that sometimes Apache will timeout even for local requests. This happens when the requests/sec is higher than 400 and/or the load is 20+.

If I retry the request, it works fine.

Here is the relevant configuration:

Timeout 300
KeepAlive On
MaxKeepAliveRequests 500
KeepAliveTimeout 2

<IfModule mpm_prefork_module>
    ServerLimit 2000
    StartServers            30
    MinSpareServers         20
    MaxSpareServers         80
    MaxRequestWorkers      2000
    MaxConnectionsPerChild   50000
</IfModule>

Some notes:

  1. The Apache error log doesn't show any errors at all.
  2. Even when this happens, the server is responsive. I can access it using SSH and other services works fine.
  3. I have munin installed. When the issue happens at least 50% the RAM is free, CPU usage is ~70% and the I/O usage is very low.

How can I get more information about this problem? What could cause this type of timeout?

  • Increase your Apache log verbosity to get useful log messages – user9517 Sep 7 '16 at 19:51
  • 1
    Would be good to distinguish between requests that hit the php app vs requests that just hit apache. Check that the server status module is enabled and hit that at times of high traffic. There is likely some contention in the php code causing some requests to be delayed, rather than something specifically in apache. – Jonah Benton Sep 7 '16 at 20:09
  • @Iain I increased the verbosity of the log but the only thing that I got was messages like this one "AH00162: server seems busy, (you may need to increase StartServers, or Min/MaxSpareServers), spawning 32 children, there are 426 idle, and 1698 total children" – Fernando Sep 12 '16 at 13:52
  • @JonahB when i check server status module I don't see a specific script or type of request that is predominant. Even during high loads time, there are 200+ idle workers and the status of each request varies a lot. The most common seems to be: Reading connection and Waiting for connection. – Fernando Sep 12 '16 at 14:18
  • 1
    Sure- the question was more in the vein of determining if only certain types of requests timeout. The expectation would be that when this problem is occurring, requests that hit the php engine would timeout, while requests that dont, that return static assets from the filesystem or just echo apache status would not timeout. Can it be confirmed that that is what happens, or do timeouts affect all requests apache sees, whether or not they are for php pages? – Jonah Benton Sep 12 '16 at 16:51
1

After a lot of tests, configuration changes etc. I couldn't fix this problem with Apache 2.4.

I decided to give Apache 2.2 a try using the same configuration. I only removed a few things that were Apache 2.4 specific directives.

It simply worked. Same server, configuration, application and load. With Apache 2.2 I don't get the timeouts anymore.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.