What could be causing memory creep on a VPS? [closed]

Question

I've got a VPS package.

I cannot log into my lxadmin because it is "out of memory".

In HyperVM I see that my VPS is using 240 MB of 256 MB.

When I reboot the HyperVM, the memory goes down to 2 MB.

But then it creeps slowly back up minute by minute, and by 15 minutes it is at 127 MB and climbing.

I basically have a couple PHP websites in subdirectories on the site that are getting about one hit per minute, these sites have quite a bit of graphics on them, do quite a bit of text parsing on every hit, and one uses an sqlite database.

From experience, what do you think could be the cause of this rapid memory creep?

What kinds of things can I do to isolate the cause?

Here's the result of top, seems like all the apache instances are adding up, what does this mean?

alt text http://tanguay.info/web/external/lxadminMemoryProblemTop.png

alt text http://tanguay.info/web/external/lxAdminMemoryProblem.jpg

Answer 1

Apache will spawn multiple processes, that is normal. You can control this in the apache configuration. The settings are dependent on what type of MPM (Multi-Processing Module) you are using.

If you are using Apache MPM worker (http://httpd.apache.org/docs/2.0/mod/worker.html) you can control it with the following settings:

ServerLimit 16 How many processes thats allowed at one time (this has to be greater or equal to MaxClients/ThreadsPerChild)

StartServers 2 How many processes to start when you start up apache

MaxClients 150 How many request you can simultaneously serve

MinSpareThreads 25 Apache will try to keep at least this many threads idle

MaxSpareThreads 75 Apache will try to keep less then this many threads idle

ThreadsPerChild 25 How many threads per process (one thread can server one request at the time)

So if you do not have enough memory to run all those processes try to lower MaxClients and ServerLimit.

---

If you are using Apache MPM prefork (http://httpd.apache.org/docs/2.0/mod/prefork.html) you can control it with the following settings:

StartServers 20 How many processes to spawn when you start up apache

MinSpareServers 10 Apache will auto adjust to keep atleast this many idle processes running

MaxSpareServers 20 Apache will auto adjust to keep less then this many idle processes

ServerLimit 150 This should be equal to MaxClient for prefork MPM

MaxClients 150 This is the maximum number of processes that apache will spawn

MaxRequestsPerChild 10000 How many request the process will handle before it gets terminated.

MaxRequestsPerChild needs some more explanation, say you have alot of diffrent PHP-scripts running and one or two of these uses alot more memory then the rest. Once one processes has run that particular script the process will stay at that memory usage until it dies. With this option you can set how often you want the processes to restart.

---

So how do you know what to tune these values too? First, figure out how much free memory you have when apache is not running. Secondly, figure out how much memory each httpd process requiers on avarage. (looks like somehwere around 15Mb from your top)

So if you have 150Mb free when apache is not running, you should limit apache to only spawn 150/15 = 10 processes.

So if you run MPM worker this might work:

ServerLimit 10  
StartServers 2 
MaxClients 150  
MinSpareThreads 25  
MaxSpareThreads 75  
ThreadsPerChild 25 

And if you are running MPM prefork this might work better:

StartServers 5  
MinSpareServers 2  
MaxSpareServers 8  
ServerLimit 10  
MaxClients 10  
MaxRequestsPerChild 1000

Answer 2

Log into an SSH shell and run top. That will give you an idea of what processes in particular are causing problems here.

Answer 3

Try using

pstree

to find out which apache process launches others. I guess you have in a site a instruction which bifurcates the process and spawn multiple instances.

Answer 4

Apache is a memory hog, I think it's normal to see apache taking that much ram with a traffic usage like yours.

My advice is to upgrade your VPS. Or if you want to get the best bang for the buck, ditch Apache and use nginx/lighttpd with php-fpm. I personally use Litespeed Webserver Standard edition on my VPS and it rocks. The only downside is that it's if your sites grow past Litespeed's limit (150 concurrent connections), you will have to pay for Enterprise Edition or switch to other webservers.

Answer 5

There are two issues here:

• The memory_limit size in PHP.
• The Apache processes that load mod_php to run PHP.

To address the first issue, you have to set the memory_limit size in php.ini, or a .htaccess file that may overwrite the defaults. The error message you see, could easily be caused by a memory limit setting of PHP, not a VPS limit.

You can easily increase the limit. Truely fixing it requires fixing the application code.

---

The second problem is with mod_php is more complex, and requires changing your Apache setup. The mod_php extension it adds memory usage to every Apache process that starts serving PHP. At first, you have pristine Apache processes that serve files, and a few that served PHP. When the amount of requests increases, more stale processes start loading PHP. At some point, you're unnecessarily consuming lots of memory.

Therefore, I'd would recommend to avoid running mod_php and run PHP as FastCGI process instead.

This way, you have control over the amount of processes that serve PHP, and all apache processes remain clean with low memory usage. My entire development server runs within 250MB, hosting websites, databases, email and redmine.

The Apache conf files need to have the following contents:

<IfModule fcgid_module>
    FcgidIPCDir /var/lib/apache2/fcgid/
    FcgidProcessTableFile /var/lib/apache2/fcgid/shm
    FcgidMaxProcessesPerClass 1
    FcgidIOTimeout 600

    AddHandler fcgid-script .fcgi

    <FilesMatch "\.php$">
        AddHandler fcgid-script .php
        Options +ExecCGI
        FcgidWrapper /srv/www/cgi-bin/php5-wrapper.sh .php
    </FilesMatch>

    # Added explicitly, by default it was added by mod_php.conf
    DirectoryIndex index.php
</IfModule>

And the php5-wrapper.sh script looks like:

#!/bin/sh
#export PHPRC=/etc/php/fastcgi/
export PHP_FCGI_CHILDREN=5
export PHP_FCGI_MAX_REQUESTS=5000
exec /usr/bin/php-cgi5

In some distributions, php-fastcgi is a separate package from php-cgi.

At some point, you can choose to go even a step further, and change the Apache worker model to threading (since PHP is left out now), or switch to a lightweight server such as nginx.