Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I'm not sure if I have a problem with a memory leak (as my hosting company suggests), or if we both need to read http://linuxatemyram.com. Maybe you clever people can help us out?

This is a front-end webserver VM running essentially only nginx & php-fpm on RHEL 5.5. This server is powering Magento, a PHP eCommerce thinggy. The server is running in a shared environment, but we're changing that soon.

Anyway.. after a reboot the server runs just fine, but within a day it will grind itself into nothingness. Pages will take literally 2 minutes to load, CPU spikes like crazy, etc.. The console is even sluggish when I SSH in. It's like my whole server is being brought to its knees.

I've also been monitoring the DB server via top and tcpdumping incoming traffic. The DB stays idle for a good portion of that "slow" load time. When i start seeing queries coming from the front-end server, the page loads soon afterward.

Here are some stats after me logging in during a slow-down, after restarting php-fpm:

[mike@front01 ~]$ free -m
             total       used       free     shared    buffers     cached
Mem:          5963       5217        745          0        192        314
-/+ buffers/cache:       4711       1252
Swap:         4047          4       4042


[mike@front01 ~]$ top
top - 11:38:55 up 2 days,  1:01,  3 users,  load average: 0.06, 0.17, 0.21
Tasks: 131 total,   1 running, 130 sleeping,   0 stopped,   0 zombie
Cpu0  :  0.0%us,  0.3%sy,  0.0%ni, 99.3%id,  0.3%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu1  :  0.3%us,  0.0%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu2  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu3  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   6106800k total,  5361288k used,   745512k free,   199960k buffers
Swap:  4144728k total,     4976k used,  4139752k free,   328480k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
31806 apache    15   0  601m 120m  37m S  0.0  2.0   0:22.23 php-fpm
31805 apache    15   0  549m  66m  31m S  0.0  1.1   0:14.54 php-fpm
31809 apache    16   0  547m  65m  32m S  0.0  1.1   0:12.84 php-fpm
32285 apache    15   0  546m  63m  33m S  0.0  1.1   0:09.22 php-fpm
32373 apache    15   0  546m  62m  32m S  0.0  1.1   0:09.66 php-fpm
31808 apache    16   0  543m  60m  35m S  0.0  1.0   0:18.93 php-fpm
31807 apache    16   0  533m  49m  30m S  0.0  0.8   0:08.93 php-fpm
32092 apache    15   0  535m  48m  27m S  0.0  0.8   0:06.67 php-fpm
 4392 root      18   0  194m  10m 7184 S  0.0  0.2   0:06.96 cvd
 4064 root      15   0  154m 8304 4220 S  0.0  0.1   3:55.57 snmpd
 4394 root      15   0  119m 5660 2944 S  0.0  0.1   0:02.84 EvMgrC
31804 root      15   0  519m 5180  932 S  0.0  0.1   0:00.46 php-fpm
 4138 ntp       15   0 23396 5032 3904 S  0.0  0.1   0:02.38 ntpd
  643 nginx     15   0 95276 4408 1524 S  0.0  0.1   0:01.15 nginx
 5131 root      16   0 90128 3340 2600 S  0.0  0.1   0:01.41 sshd
28467 root      15   0 90128 3340 2600 S  0.0  0.1   0:00.35 sshd
32602 root      16   0 90128 3332 2600 S  0.0  0.1   0:00.36 sshd
 1614 root      16   0 90128 3308 2588 S  0.0  0.1   0:00.02 sshd
 2817 root       5 -10  7216 3140 1724 S  0.0  0.1   0:03.80 iscsid
 4161 root      15   0 66948 2340  800 S  0.0  0.0   0:10.35 sendmail
 1617 nicole    17   0 53876 2000 1516 S  0.0  0.0   0:00.02 sftp-server
 ...

Is there anything else I should be looking at, or any more information that might be useful? I'm just a developer, but the slowdowns on this system worry me and make it hard to do my work..

Help me out, ServerFault!

share|improve this question
    
What's your VM's hypervisor? VMware ESX? Xen? VirtualBox? –  raspi Dec 15 '10 at 20:58
    
It's VMware ESX, I believe.. but I'm not 100% certain, raspi. The VMs are being provided by our ISP. They provide our head office an internet connection, provide VPN to our brick & mortar stores, colo our existing web server, etc.. Does it matter? –  incredimike Dec 15 '10 at 21:43
    
free -m claims that you're using 4.5G of RAM, but nothing in your top output backs that up (I'm looking at the %MEM column); is there anything else running (i.e. have you elided the output of top)? –  nickgrim Mar 27 '11 at 23:53
    
The output was not edited.. it's actually being displayed from % mem use descending. As Stuart suggests below, I think the problem was a memory leak.. and I assume TOP's % used wouldn't show a "leak" even though "free -m" might? –  incredimike Nov 22 '12 at 19:07

4 Answers 4

In case anyone else suffers this.

We have just experienced the same issue. A memory leak in php5-fpm. The RAM is used up with every page request and eventually maxed out. Then the CPU goes into overdrive with the KSWAP process running the swap disk.

the only thing that fixed, although not the ideal setup was to change our www.conf pool file

pm = dynamic

to

pm = ondemand

memory now seems stable.

share|improve this answer
    
One year later, how did this work out? Did you end up keeping this configuration? –  Mahn Oct 29 '13 at 12:46

The next time the server slows down, run 'vmstat 1' and 'iostat 1', then report the results back to us.

share|improve this answer

My guess is that php5 just got updated with the default memory limit of 128Mb and there are too many php5 instances running ( check pm.max_children in your php-fpm config ).

The above happened to mee too, and I've been pulling my hair trying to figure it out. I have logged a bugreport with ubuntu, but I assume it needs to be fixed upstreams with php-fpm -- the defaults I had would eat up 6Gb RAM if it could. And that sure as hell made my 1Gb server useless...

share|improve this answer

You probably don't have some opcode cache installed on the server which is causing the slowness.

Moreover, as Marco said, you might be running a large number of php-fpm instances on the system.

In case you suspect a memory leak trouble, install suhosin extension for php, which will prevent memory leaks.

Actually if it is a memory leak it should be logged in the php error log, php.ini has an option to report memleaks, report_memleaks = On

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.