Apache log rotation: logrotate vs rotatelogs vs chronolog

Question

I have been researching log rotation for my server which hosts ~5 fairly high traffic sites. From what I can tell, my options are to use logrotate or to use piped logging with either rotatelogs or chronolog.

logrotate requires a restart of apache and both SIGHUP and SIGUSR1 restarts are less than ideal on high traffic sites, because either you drop a bunch of connections or you need to delay compressing the old log until all child processes have died naturally. Also, downtime can be quite significant if compression is enabled. Would using logrotate - without compression and with graceful restart - and compressing old logs after the fact be the best way to minimize downtime?

chronolog and rotatelogs sound promising, but are not well documented. I couldn't find examples of using either in combination with vhost specific logs. The chronolog website says, "when the expanded filename changes, the current file is closed and a new one opened". Is this globally? Or is that per AccessLog, CustomLog or ErrorLog directive?

Is there a significant difference between chronolog and rotatelogs?

Answer 1

I suppose having logs written to file by apache is preferred solution in terms of stability. Imagine what might happen if your syslog will go down, or if there'll be some error in pipe command?

Why not to write your own rotation script? You could use following algorithm to avoid losing data and any downtimes.

Suppose you need to rotate file big_access_log

1. rename big_access_log to .big_access_log_rotating
2. send HUP signal to apache. It will reopen (recreate) big_access_log.
3. compress .big_access_log_rotating with any tool you like at any priotiry.

Personally, I just use logrotate and I don't care a lot about some data that might be lost, but I suppose logrotate is smart enough to do rotation in atomic way.

Answer 2

If you set Apache to log via syslog and also use rsyslog to handle logging, you could configure rsyslog to send your logs to a file that's named according to the current date. That would eliminate the need to restart Apache, and you could easily rotate the older log files away.

Check the following sites for more info:

http://wiki.rsyslog.com/index.php/Working_Apache_and_Rsyslog_configuration

http://wiki.rsyslog.com/index.php/DailyLogRotation

Answer 3

Another thing you could do is set up cronjob that does the following:

/usr/bin/find /path/to/logs -name access.log | \
    /usr/bin/sort | \
    /usr/bin/head --lines=-1 | \
    /usr/bin/xargs -r /usr/bin/bzip2 -9

This assumes that the uncompressed access logs can be sorted in chronological order easily. The last line (which would be the current log) is left out, and the remaining logfiles are compressed. If there are none, the -r flag on xargs means the command won't be run.

This can be tuned further to compress multiple logs at the same time if need be