1

For some reason, most Sundays around 5-6AM local time Apache is not working properly. I can't find the reason, and I'm stuck troubleshooting.

I'm running Magento on Apache2 with an SSL certificate, and around that specific time random pages stop functioning resulting in no orders being placed. Sometimes it's the category page, sometimes it's the admin area.

I'm thinking it may be logrotating, but not sure how to troubleshoot further. Here's my Apache error log for the past two Sundays:

Jan 29th

/var/log/apache2/error.log.1:[Sun Jan 29 00:35:21 2012] [error] server reached MaxClients setting, consider raising the MaxClients setting

/var/log/apache2/error.log.1:[Sun Jan 29 00:39:44 2012] [error] [client 66.87.100.194] request failed: error reading the headers

/var/log/apache2/error.log.1:[Sun Jan 29 06:25:21 2012] [notice] Graceful restart requested, doing restart

--- notice new log file

/var/log/apache2/error.log:[Sun Jan 29 06:25:24 2012] [notice] Apache/2.2.16 (Debian) PHP/5.3.3-7+squeeze1 with Suhosin-Patch mod_ssl/2.2.16 OpenSSL/0.9.8o configured -- resuming normal operations

/var/log/apache2/error.log:[Sun Jan 29 06:26:00 2012] [notice] child pid 7301 exit signal Segmentation fault (11)

Jan 22nd

/var/log/apache2/error.log.2.gz:[Sun Jan 22 06:25:17 2012] [notice] Graceful restart requested, doing restart

--- notice new log file

/var/log/apache2/error.log.1:[Sun Jan 22 06:25:22 2012] [notice] Apache/2.2.16 (Debian) PHP/5.3.3-7+squeeze1 with Suhosin-Patch mod_ssl/2.2.16 OpenSSL/0.9.8o configured -- resuming normal operations

/var/log/apache2/error.log.1:[Sun Jan 22 16:26:11 2012] [notice] caught SIGTERM, shutting down

/var/log/apache2/error.log.1:[Sun Jan 22 16:26:12 2012] [notice] Apache/2.2.16 (Debian) PHP/5.3.3-7+squeeze1 with Suhosin-Patch mod_ssl/2.2.16 OpenSSL/0.9.8o configured -- resuming normal operations

/var/log/apache2/error.log.1:[Sun Jan 22 16:26:14 2012] [notice] child pid 30367 exit signal Segmentation fault (11)

3 Answers 3

1

Not quite a programming problem, but there's this

server reached MaxClients setting, consider raising the MaxClients setting

which indicates Apachex is trying to take on more requests (at once) than it's configured for. Upping this number may be a good idea.

There's also these two lines

/var/log/apache2/error.log:[Sun Jan 29 06:25:24 2012] [notice] Apache/2.2.16 (Debian) PHP/5.3.3-7+squeeze1 with Suhosin-Patch mod_ssl/2.2.16 OpenSSL/0.9.8o configured -- resuming normal operations

/var/log/apache2/error.log.2.gz:[Sun Jan 22 06:25:17 2012] [notice] Graceful restart requested, doing restart

Which indicate someone is upgrading server packages, which may interfere with performance of the server itself.

ServerFault's a good place to ask a question like this.

A better place would be your ISP/IT-Department/Web-Host.

1
  • I don't believe maxclients is the issue as it doesn't appear in the log on every crash. Also, it seems to be a notice, and not an error causing unresponsiveness at the point in time.
    – Rune
    Feb 1, 2012 at 9:39
1

Do you have any cronjobs running at that same time? My hunch is that there is a system update running that updates some shared library loaded by your apache app, which will often cause segfaults.

3
  • +1 My hunch is that the Sunday "graveyard" is a slot where the sysadmins are doing backups and bouncing Apache. Talk to the IT support group, and understand the constraints here that you need to work with.
    – TerryE
    Jan 31, 2012 at 13:19
  • This seems likely actually. Going to contact my hosting provider and ask. Will let you know.
    – Rune
    Feb 1, 2012 at 9:38
  • Apparently backups etc is running 5 hours before. It's the exact same time as logrotate etc., but it's running daily so I don't see why it should only have an impact on Sunday?
    – Rune
    Feb 1, 2012 at 12:23
0

I did research on google about "segmentation fault" error, most of them related with 3rd party applications which is can be eAccelerator, iconcube, zend encoder etc. Also, somebodies written a comment that can be related with PHP edition. Their suggestion, uninstall the whole PHP then re-install again.

My suggestion, if you have eAccelerator, remove for a while and look up the log again. If error keep going, try to install fresh PHP without "Suhosin" patch.

Segmentation Fault

8
  • Would segmentation fault result in unresponsive pages as I experience? I'm not using eAccelerator and reluctant to re-install PHP on a live server.
    – Rune
    Feb 1, 2012 at 12:25
  • I've found that APC can cause segmentation faults. 2bits.com/articles/… under "Drawbacks of PHP op-code caches: Segmentation Faults". Could that explain my issues?
    – Rune
    Feb 1, 2012 at 13:12
  • @Rune as I said in my post, you should try step by step if you are not sure which application has side effects. The interesting part in your issue "the problem raise in specified times", that is interesting. For instance, you can check Linux log file from related days like vi /var/log/messages, vi /var/log/cron.log Feb 1, 2012 at 15:55
  • I'll try to narrow it down. For now I've tweaked APC. I've looked through the logs and all I find around that specific time is this (linking to pastebin) pastebin.com/ZctmSLjr
    – Rune
    Feb 2, 2012 at 11:01
  • check this link : CRON: Regular Tasks Feb 2, 2012 at 11:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .