Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

OK, I hate to be the helpless Apache noob, but I am feeling stumped here.

All of a sudden last night, our WordPress site went down. I rebooted it and watched for a couple of minutes and it seemed all right, so I left it alone. Then I wake up and find it's down again. After a little investigation, I've discovered that, despite only getting 20 or so requests per minute at the time, Apache keeps forking a new instance for just about every request until it hits MaxClients, and then the instances just sit there doing nothing. Literally 0.1% CPU utilization for the whole system at that point. If I log into MySQL and look at the process list, I can see a corresponding database connection for each httpd, so it looks like the scripts are never ending. But if I request a static file or even a simple "Hello world" PHP file before it reaches MaxClients, that request will go through fine.

I'm really at a loss as to even what to look at, because nobody else here has the technical sophistication to SSH into the box or even install plugins, and I know I haven't touched it in days at least — so I don't even know what could have changed to cause the problem.

The setup is Apache 2.2.3/prefork with mod_php 5.2.6. Here are the obviously relevant settings (let me know if you any to know anything else):

httpd.conf

Timeout 20
KeepAlive Off
<IfModule prefork.c>
    StartServers 2
    MinSpareServers 1
    MaxSpareServers 3
    MaxClients 50
    MaxRequestsPerChild  2000
</IfModule>

php.ini

max_execution_time = 600 ; Set so high for large file uploads
max_input_time = 600 ; Set so high for large file uploads
memory_limit = 128M ; Set so high for large file uploads

log_errors = On

A few things I've tried:

  • Upping MaxClients
    • This just resulted in Apache eating up all 1.6 GB of RAM and then doing the same thing as before
  • Cutting max_execution_time and max_input_time to 15 and memory_limit to 32M
    • Made no difference — the httpd instances were still immortal
  • Reinstalling WordPress
    • No difference at all
  • tail -fing error_log
    • No errors reported aside from reaching MaxClients
  • tail -fing access_log to see if we're being DDOSed or something
    • Traffic was indeed pretty low while this was happening

I feel like I must be missing something right in front of my face, but for the life of me I cannot figure out what is going wrong here. So I'm hoping someone a little more experienced in sysadminning has seen whatever I'm doing wrong before.

share|improve this question
    
Did it starting swapping to a standstill? What plugins for WP are you using? Some attempt to increase the allowed memory for PHP (especially those that have graphics functionality). Though the memory is not always committed, it may lead to a situation where the system is swapping to a standstill. –  0xC0000022L May 19 '12 at 4:24

2 Answers 2

up vote 0 down vote accepted

Your php scripts in Apache suddenly became very slow. You need to find out what is your bottleneck. Since you have no high cpu usage - it can be swapping, slow disk or network IO, slow database server, slow dns requests - and many other reasons.

First check all logs for suspicious errors. If you still have no idea - you can try to strace apache process to see which call is slow as a start to debugging.

share|improve this answer

Ok sorry forthe keepalives question.. What does "show processlist" says on the mysql server? Typically a locked/corrupted table would explain the issue.

share|improve this answer
    
Keepalives ARE set to "off" - look at the httpd.conf –  Henk May 19 '12 at 8:01

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.