Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Every hour or so an Apache child process seg. faults on our web server. We are running a non-threaded PHP 5.2.17 Apache module with Apache prefork MPM. I have run httpd with a few core dumps, gdb and this .gdbinit file from php's github repo, typing this command into gdb for each:

dump_bt executor_globals.current_execute_data

With no experience with gdb or the inner-workings of the php interpreter, I can't make anything from the results.

core.22762
[0x53896ef0] () :-2118682552
[0x538977a0] () /Statement/Interface.php:113
[0x538978a0] /Zend/Db/Statement/Interface.php()

core.22791
[0x538977a0] () @:0
[0x538978a0] ()

core.5568
[0x53896ef0] () :2061035360
[0x538977a0] () :1767992432
[0x538978a0] ()

core.30384
[0x538977a0] () :0
[0x538978a0] ()

core.3091
[0x53896ef0] mysql_query():992424253
[0x538977a0] () ~:17
[0x538978a0] ()

(core.3091 also shows this after Program terminated with signal 11, Segmentation fault.)

#0  0x00002b6e7ad8d67d in zend_do_fcall_common_helper_SPEC (execute_data=0x7fff53896ef0) at /usr/src/debug/php-5.2.17/Zend/zend_vm_execute.h:217
217                             EX_T(opline->result.u.var).var.fcall_returned_reference = return_reference;

Is there something else that I can type in gdb to get a better stack trace? Is there anything else that you do when debugging Apache segmentation faults?

Any help would be greatly appreciated, thanks.

share|improve this question
1  
Running bt full from (gdb) prompt, it may helpful. But if you're not familiar with PHP, all you can do is file a bug to PHP developers. –  quanta Aug 5 '11 at 14:03
    
Thanks for the tip. bt full definitely gives more information. I'll probably end up filing a bug on the PHP site. –  weberwithoneb Aug 5 '11 at 17:02
add comment

1 Answer

I've seen this before.. apart from getting your PHP app developers to check their code, and making sure you have up-to-date versions (from your distribution) of PHP and apache, there isn't much you can do.

If you're running an Op-Code cacher (eaccelerator, APC or XCache), you could try turning them off, they're known to cause the odd segfault.

In the past, we've had to deploy some impressively hackish solutions to a segfaulting system, for short-term fixes, of course. For example, this cronjob worked well:

# Restart apache when a segfault is found in the most recent line of errorlog #*/5 * * * * tail -n1 /var/log/apache2/error.log | grep 'Segmentation fault' && /etc/init.d/apache2 restart

Or you could try something a bit more clever: #*/5 * * * * /usr/bin/wget http://www.my-site.com/ -T10 -O 2> /dev/null - | grep "Hosted by" > /dev/null || /usr/local/bin/brutally_restart_apache.sh

Where brutally_restart is:

    #!/bin/sh
    /usr/sbin/apache2ctl stop
    sleep 6
    killall -9 apache2
    sleep 4
    /usr/sbin/apache2ctl start
share|improve this answer
    
"brutally_restart_apache.sh" - nice name ;) Unfortunately, disabling APC didn't help. We have something similar in place for restarting Apache. We're still on PHP 5.2.17, so hopefully upgrading to 5.3 will help. –  weberwithoneb Aug 11 '11 at 17:51
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.