Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have two servers - one runs a PHP webservice, and the other is the database server. I have high load (~9.0), and I believe it is a configuration issue of some sort. The PHP processing is minimal, all the heavy lifting is done by the database server. I have approximately 6-8 requests per second.

System is CentOS/Plesk (not my choice).

top looks like this:

top - 19:54:38 up  4:38,  2 users,  load average: 9.42, 9.10, 8.86
Tasks:  48 total,  12 running,  36 sleeping,   0 stopped,   0 zombie
Cpu(s): 67.4%us, 32.5%sy,  0.0%ni,  0.2%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   2097152k total,   231120k used,  1866032k free,        0k buffers
Swap:        0k total,        0k used,        0k free,        0k cached   

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                                                                   
11806 apache    16   0 50416  28m 4744 R 24.6  1.4   7:27.61 httpd                                                                                                                                                      
11770 apache    16   0 49140  27m 4764 R 23.6  1.4   7:32.80 httpd                                                                                                                                                      
11783 apache    16   0 50424  28m 4736 R 23.2  1.4   7:19.99 httpd                                                                                                                                                      
30217 apache    16   0 50412  28m 4732 R 21.6  1.4   3:33.00 httpd                                                                                                                                                      
26245 apache    16   0 50412  28m 4732 R 21.2  1.4   4:14.98 httpd                                                                                                                                                      
26256 apache    16   0 48880  27m 4732 R 21.2  1.3   4:15.62 httpd                                                                                                                                                      
13659 apache    15   0 52464  30m 4764 R 19.2  1.5   5:23.12 httpd                                                                                                                                                      
26255 apache    16   0 48880  27m 4748 R 17.3  1.3   4:16.62 httpd                                                                                                                                                      
30215 apache    16   0 48796  27m 4732 R 16.3  1.3   3:45.32 httpd                                                                                                                                                      
29703 apache    16   0 52208  30m 4764 R 12.3  1.5   3:46.21 httpd    

Load on the database server is next to nothing - 0.05-0.07.

I'm not a server expert so I've been messing with this issue for a while now. I thought I fixed it a while back by running a yum update on the server - many packages updated, so I presumed it was a fixed bug in httpd or similar. The issue returned, I ran yum update again, the issue went away again. Now the issue is back and updating does not fix it - I find this whole thing quite curious.

netstat has many (~100) identical entries like this (IPs and hostnames obscured for NDA reasons):

Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State      
tcp        0      0 123.123.123.123:12345       mydbserver.com:mysql     TIME_WAIT

In addition it has another hundred or so entries for incoming connections.

I would appreciate any ideas or ways to diagnose this issue - I'm not a server guy primarily (I'm a programmer) and have been dropped in the deep end on this one. I'm reasonably certain the PHP webservice is not to blame - the fact that at one point the issue was resolved makes me feel that it is fine and it's a system config issue.

share|improve this question
add comment

1 Answer 1

  • Check DNS(# ping mydbserver.com)
  • Check MySQL PROCESSLIST(mysql> SHOW FULL PROCESSLIST)
  • Use strace(# strace -fp $APACHE_PID), to profile(# strace -cfp $APACHE_PID)
  • If use PHP-FPM, use slowlog
  • Check iptables rulues and network connectivity with MySQL server
  • Use microtime() to log your PHP script or use Pinba. Use profile tools
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.