14

Every time logrotate runs Apache/2.4.7 (Ubuntu) experiences a seg fault and does not restart:

[Wed Sep 10 06:35:54.266018 2014] [mpm_event:notice] [pid 20599:tid 140630283466624] AH00493: SIGUSR1 received.  Doing graceful restart
[Wed Sep 10 06:35:54.885118 2014] [core:notice] [pid 20599] AH00060: seg fault or similar nasty error detected in the parent process

My apache logrotate script looks like:

/var/log/apache2/*.log {
    daily
    missingok
    rotate 52
    compress
    delaycompress
    notifempty
    create 640 root adm
    sharedscripts
    postrotate
        /usr/sbin/apachectl graceful
    endscript
    prerotate
        if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
            run-parts /etc/logrotate.d/httpd-prerotate; \
        fi; \
    endscript
}

/srv/apache/log/*.log {
    daily
    missingok
    rotate 52
    compress
    delaycompress
    notifempty
    create 640 root adm
    sharedscripts
    postrotate
        /usr/sbin/apachectl graceful
    endscript
    prerotate
        if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
            run-parts /etc/logrotate.d/httpd-prerotate; \
        fi; \
    endscript
}

I keep the default logs in /var/log/apache2, but I keep vhost specific logs (for the three different vhosts hosted on this server) in the /srv/apache/log directory (e.g. mysite1_error.log, mysite1_access.log, mysite2_error.log, mysite2_access.log....).

The relevant parts of /etc/apache2/apache.conf:

#/etc/apache2/apache.conf
ErrorLog ${APACHE_LOG_DIR}/error.log
LogLevel trace8   # I set to try and get more info about this problem.
LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined
LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %O" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent

where export APACHE_LOG_DIR=/var/log/apache2$SUFFIX. In my vhost conf files, such as /etc/apache2/sites-enabled/mysite1.conf I have:

#/etc/apache2/sites-enabled/mysite1.conf
LogLevel debug
ErrorLog  /srv/apache/log/mysite1_error.log
CustomLog /srv/apache/log/mysite2_access.log combined

and similar for the other sites.

Anyone know why logrotate is causing this crash?

One more thing:

Manually forcing the logrotate as root:

logrotate -v -f /etc/logrotate.d/apache2

does not cause the seg fault, but I know it's logrotate as I've tried playing with the times (weekly, daily) and the seg fault always occurs exactly as the logs are being rotated only.

How to reproduce on demand

# Set a crontab to run each minute (simulating cron.daily run of logrotate)
crontab -e
*/1 * * * * root /usr/sbin/logrotate -v -f /etc/logrotate.d/apache2 > /home/myuser/logrotate.log 2>&1

Finally

If I comment out the /srv/apache/log/*.log rotation block in /etc/logrotate.d/apache the seg fault does not occur either.

  • Form the error message it looks like /usr/sbin/apachectl graceful is the offender (in theory scripts within /etc/logrotate.d/httpd-prerotate are also candidates). Can you confirm by running /usr/sbin/apachectl graceful manually? Is there any more hints when this happens in Apache's global errorlog (or the global syslog), maybe when you increase the LogLevel not only within <VirtualHost> but globally? – Nils Toedtmann Sep 10 '14 at 11:06
  • /usr/sbin/apachectl graceful manually at cmd line causes no problems (Also note that this line was originally /etc/init.d/apache2 reload > /dev/null but after getting the seg fault, I changed to apachectl on advice I read online, obviously it did not fix things). The LogLevel is already at the highest possible globally, trace8 set in the apache.conf file. – fpghost Sep 10 '14 at 11:32
  • And just to rule it out: /etc/logrotate.d/httpd-prerotate does not exist. – fpghost Sep 10 '14 at 11:33
  • This is usually caused by a module not shutting down properly. Are you using any exotic modules? Anything added recently? 3rd-party modules? Try disabling them. – Giovanni Tirloni Sep 10 '14 at 12:18
  • So /usr/sbin/apachectl graceful is causing a segfault when issued by logrotate, but not when issued manually? Odd – Nils Toedtmann Sep 10 '14 at 12:41
12

It seems that if I just change the logrotate script to be a single block, i.e. /var/log/apache2/*.log, /srv/apache/logs/*.log {....} I don't get the segfault. So it was just the two blocks were causing a second restart attempt during the first restart...

If I enter apache2ctl graceful & apache2ctl graceful & apache2ctl graceful & apache2ctl graceful & on the cmd line my system DOES segfault, which seems to confirm

  • 1
    I have this exact problem, but only one block in my logrotate script to begin with. Any ideas? – kontextify Dec 28 '18 at 9:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.