I have an odd one that I just noticed yesterday. Over a 24 hour period, the cpu load on our apache machines (16 dual hex core machines) doubles. We handle about 8 billion requests per month, and everything sits behind haproxy load balancers. The number of running processes on each machine is always the same (capped in the load balancer), and any queue builds in haproxy. We limit the number of running sessions in haproxy to 90 per machine (chosen because of ram and cpu limits on each machine)
We're currently using the event mpm with the following config:
event MPM <IfModule event.c> ServerLimit 15 MaxClients 960 StartServers 2 ThreadsPerChild 64 ThreadLimit 64 MinSpareThreads 32 MaxSpareThreads 112 MaxRequestsPerChild 20000 </IfModule>
At the current rate, 20K requests should mean the thread is restarted every 2 minutes or so.
Memory usage is low. We have 12gb of ram on each machine, and at peak they only use about 7gb. A restart of apache immediately drops the load in half, and it stays that way for several hours.
Any ideas or additional info needed?