FastCGI and Apache 500 error intermittently

Question

I have a FastCGI (mod_fastcgi)problem. It happens every once in a while, and does not casue a complete server meltdown, just 500 errors. Here are a couple things. First I am using APC so PHP is in control of it's own processes, not FastCGI. Also, I have the webroot set as:

/var/www/html

And the fcgi-bin inside:

/var/www/html/fcgi-bin

First off here is the apache error_log:

[Fri Jan 07 10:22:39 2011] [error] [client 50.16.222.82] (4)Interrupted system call: FastCGI: comm with server "/var/www/html/fcgi-bin/php.fcgi" aborted: select() failed, referer: http://www.domain.com/

I also ran strace on the 'fcgi-pm' process. Here is a snip from the trace around the time it bombs out:

21725 gettimeofday({1294420603, 14360}, NULL) = 0
21725 read(14, "C /var/www/html/fcgi-bin/php.fcgi - - 6503 38*", 16384) = 46
21725 alarm(131)                        = 0
21725 select(15, [14], NULL, NULL, NULL) = 1 (in [14])
21725 alarm(0)                          = 131
21725 gettimeofday({1294420603, 96595}, NULL) = 0
21725 read(14, "C /var/www/html/fcgi-bin/php.fcgi - - 6154 23*C /var/www/html/fcgi-bin/php.fcgi - - 6483 28*", 16384) = 92
21725 alarm(131)                        = 0
21725 select(15, [14], NULL, NULL, NULL) = 1 (in [14])
21725 alarm(0)                          = 131
21725 gettimeofday({1294420603, 270744}, NULL) = 0
21725 read(14, "C /var/www/html/fcgi-bin/php.fcgi - - 5741 38*", 16384) = 46
21725 alarm(131)                        = 0
21725 select(15, [14], NULL, NULL, NULL) = 1 (in [14])
21725 alarm(0)                          = 131
21725 gettimeofday({1294420603, 311502}, NULL) = 0
21725 read(14, "C /var/www/html/fcgi-bin/php.fcgi - - 6064 32*", 16384) = 46
21725 alarm(131)                        = 0
21725 select(15, [14], NULL, NULL, NULL) = 1 (in [14])
21725 alarm(0)                          = 131
21725 gettimeofday({1294420603, 365598}, NULL) = 0
21725 read(14, "C /var/www/html/fcgi-bin/php.fcgi - - 6179 33*C /var/www/html/fcgi-bin/php.fcgi - - 5906 59*", 16384) = 92
21725 alarm(131)                        = 0
21725 select(15, [14], NULL, NULL, NULL) = 1 (in [14])
21725 alarm(0)                          = 131
21725 gettimeofday({1294420603, 454405}, NULL) = 0

I noticed that the 'select()' seems to stay the same regardless, however the read() changes its return from 46 to some other number while it is bombing out. Has anyone seen anything like this. Could this be some sort of file locking?

Thanks, Ben

Answer 1

Thanks for your response. I have all of the PHP error going to a log file. I get a few notices but no errors. I must admit that I did not write this code. For now I have redirected all 500 errors to the index.php, using a '.htaccess' rule. I must be missing something though. This should not be happening. The only guess I have is that once the 'PHP_FCGI_MAX_REQUESTS' reaches it max, php kills the child and this confuses the FastCGI. However, if I understand correctly PHP has a parent process that should be the only one that FastCGI talks to, so I am not to sure that that is it... Here is my wrapper script:

#!/bin/bash
# Shell Script To Run PHP5 using mod_fastcgi under Apache 2.x
# Tested under Red Hat Enterprise Linux / CentOS 5.x
### Set PATH ###
PHP_CGI='/usr/bin/php-cgi -d apc.shm_size=60M'
PHP_FCGI_CHILDREN=25
PHP_FCGI_MAX_REQUESTS=1000
### no editing below ###
export PHP_FCGI_CHILDREN
export PHP_FCGI_MAX_REQUESTS
exec $PHP_CGI

This is very high volume server so that is why the PHP_FCGI_CHILDREN is set so high.

Thanks again, Ben

Answer 2

I read somewhere (dealing with lighttpd, not apache) that php cannot handle more than 500 requests for some reason. The 501st request will bomb for whatever reason.

Sorry I do not have more information than that, but it's at the very least worth a shot.

tl;dr try setting PHP_FCGI_MAX_REQUESTS to 500 and seeing if the problem clears itself up.

Found the information, it applies to Lighttpd, and I do not know if it applies to apache or not.

Test it and I would love to hear if this is only an issue with lighttpd, or if it is a general issue.

Why is my PHP application returning an error 500 from time to time?

"This problem seems to stem from a little-known issue with PHP: PHP stops accepting new FastCGI connections after handling 500 requests; unfortunately, there is a potential race condition during the PHP cleanup code in which PHP can be shutting down but still have the socket open, so lighty can send request number 501 to PHP and have it "accepted", but then PHP appears to simply exit, causing a 500 return from lighty.

To limit this occurance, set PHP_FCGI_MAX_REQUESTS to 500."

--http://redmine.lighttpd.net/projects/1/wiki/Docs:PerformanceFastCGI