Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

We have a webserver with apache 2.2.14, PHP 5.3.2.

PHP is executed using mod_fcgid (see bottom). Everything works fine but sometimes, and we still have to figure out what triggers this, when php processes are "rotated" they remain active and orphaned: apache spawns new php processes and the old ones remains on the system. Killing them doesn't always kicks them away. More likely an "apache2ctl graceful" frees the system from this stale processes. We found this in error log: [Tue Jun 18 20:49:54 2013] [warn] mod_fcgid: process 2009 graceful kill fail, sending SIGKILL from what I found searching around this is quite normal, but that's all I found in apache logs during one of this processes leak.

This event is luckily seldomly happening, usually apache and php operate normally and with no problems during fcgid childrens renewal. How can we understand what's going wrong in these situations?

mod_fcgid config in site:

<IfModule mod_fcgid.c>
SuexecUserGroup domain domain
<Directory /var/www/fomain.it/htdocs/>
AddHandler fcgid-script .php
FCGIWrapper /var/www/fcgi/domain.it/fcgi-starter-php .php
Options +ExecCGI -Indexes
AllowOverride FileInfo Options
Order allow,deny
Allow from all
</Directory>
<Directory /var/www/fcgi/domain.it/>
AllowOverride None
Options +ExecCGI MultiViews -Indexes
Order allow,deny
Allow from all
</Directory>
</IfModule>

/var/www/fcgi/domain.it/fcgi-starter-php:

#!/bin/sh

PHPRC=/var/www/fcgi/domain.it/php/

export PHPRC
PHP_FCGI_CHILDREN=8
export PHP_FCGI_CHILDREN
PHP_FCGI_MAX_REQUESTS=5000
export PHP_FCGI_MAX_REQUESTS

exec /usr/lib/cgi-bin/php $1

We currently found a workaround to avoid the server to fill up with orphaned processes. You can get rid of the stale processes with:

apache2ctl graceful

and kill those processes with:

pkill -f -x /usr/lib/cgi-bin/php -P 1

scripting and scheduling those two commands (with proper checks) will avoid the server to host plenty of useless procs, but the issue is still present.

share|improve this question
    
i have the same problem – user1179459 Mar 18 '14 at 1:57
up vote 0 down vote accepted

I had a similar problem that fcgid ran out of available process slots after some period of activity.

The log message was roughly:

[fcgid:warn] mod_fcgid: can't apply process slot for /var/www/cgi-bin/xxx/php-cgi, referer: ...

I tracked the issue down to this:

[fcgid:emerg] (35)Resource deadlock avoided: [client ....] mod_fcgid: can't get pipe mutex, referer: ...

which gets caused by improper locking. In my case, Apache used fcntl() locking (by default on debian), so I changed that to flock() in apache2.conf:

Mutex flock:${APACHE_LOCK_DIR} default

Reference that led me to the solution: https://bz.apache.org/bugzilla/show_bug.cgi?id=53999

Documentation about various locking options (fcgid has a warning advice against usign it with anything that would involve threads): https://httpd.apache.org/docs/2.4/mod/core.html#mutex

share|improve this answer
    
thanks for your feedback. I'll accept the answer even if I cannot test it anymore, having happily switched to nginx+php-fpm long time ago. Thanks again for reporting back! – Maxxer Mar 11 at 10:39

I don`t have a real answer, but maybe more information can help to solve the problem. I like to tell, I have the same problem on a windows server with PHP 5.3.5.

Some cgi-processes remain as a kind of zombie task, after the done their real execution. They even ignore settings like max_execution time.

Currently I have a scheduled script what kills these old processes to. problem with this solution is, even "normal" cgi processes where killed, maybe it would be usefull to detect the runtime of the processes and only kill them if they have expired their max_execution time.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.