OK, I hate to be the helpless Apache noob, but I am feeling stumped here.
All of a sudden last night, our WordPress site went down. I rebooted it and watched for a couple of minutes and it seemed all right, so I left it alone. Then I wake up and find it's down again. After a little investigation, I've discovered that, despite only getting 20 or so requests per minute at the time, Apache keeps forking a new instance for just about every request until it hits MaxClients, and then the instances just sit there doing nothing. Literally 0.1% CPU utilization for the whole system at that point. If I log into MySQL and look at the process list, I can see a corresponding database connection for each httpd, so it looks like the scripts are never ending. But if I request a static file or even a simple "Hello world" PHP file before it reaches MaxClients, that request will go through fine.
I'm really at a loss as to even what to look at, because nobody else here has the technical sophistication to SSH into the box or even install plugins, and I know I haven't touched it in days at least — so I don't even know what could have changed to cause the problem.
The setup is Apache 2.2.3/prefork with mod_php 5.2.6. Here are the obviously relevant settings (let me know if you any to know anything else):
httpd.conf
Timeout 20
KeepAlive Off
<IfModule prefork.c>
StartServers 2
MinSpareServers 1
MaxSpareServers 3
MaxClients 50
MaxRequestsPerChild 2000
</IfModule>
php.ini
max_execution_time = 600 ; Set so high for large file uploads
max_input_time = 600 ; Set so high for large file uploads
memory_limit = 128M ; Set so high for large file uploads
log_errors = On
A few things I've tried:
- Upping MaxClients
- This just resulted in Apache eating up all 1.6 GB of RAM and then doing the same thing as before
- Cutting max_execution_time and max_input_time to 15 and memory_limit to 32M
- Made no difference — the httpd instances were still immortal
- Reinstalling WordPress
- No difference at all
tail -fing error_log- No errors reported aside from reaching MaxClients
tail -fing access_log to see if we're being DDOSed or something- Traffic was indeed pretty low while this was happening
I feel like I must be missing something right in front of my face, but for the life of me I cannot figure out what is going wrong here. So I'm hoping someone a little more experienced in sysadminning has seen whatever I'm doing wrong before.
