Server Fault is a question and answer site for system and network administrators. It only takes a minute to sign up.
Sign up to join this community
I've been told that our (scripted) daily reboot of apache has been set up because of our log rotation. The concern is that logrotate might lose a line in the access logs if it rotates while apache is running and accepting requests.
Can someone tell me if this is plausible, true, or complete nonsense?
You need to restart the server after the logs have been rotated as Apache holds open the file handle to the log and will continue writing to the old log file if you don't. Sending USR1 to the parent (graceful restart) should do the trick.
Quote from 2.2 docs:
On even a moderately busy server, the quantity of information stored in the log files is very large. The access log file typically grows 1 MB or more per 10,000 requests. It will consequently be necessary to periodically rotate the log files by moving or deleting the existing logs. This cannot be done while the server is running, because Apache will continue writing to the old log file as long as it holds the file open. Instead, the server must be restarted after the log files are moved or deleted so that it will open new log files.
By using a graceful restart, the server can be instructed to open new log files without losing any existing or pending connections from clients. However, in order to accomplish this, the server must continue to write to the old log files while it finishes serving old requests. It is therefore necessary to wait for some time after the restart before doing any processing on the log files. A typical scenario that simply rotates the logs and compresses the old logs to save space is:
mv access_log access_log.old
mv error_log error_log.old
apachectl graceful sleep 600
gzip access_log.old error_log.old
Another way to perform log rotation is using piped logs as discussed in the next section.
Totally plausible. Depends on how you are stopping Apache.
If you are doing a nightly HUP to get the servers to start the new log then you should not have a problem. Any clumsy stop and restart and your own your own though!
However, any connections present during the nightly HUP will be lost and those punters will have to reconnect. Bit of a pity if they're in the middle of a big download!
If you're logging the bytes served as a part of the request, then the connection isn't logged until the download is complete, hence those connections that were dropped because of the HUP will be missing from the logs.
Have you looked at the new 2.2.12 release?
One of the changes is to make the "piped through rotatelogs" more robust so that you can defer the rotation out to the rotatelogs utility and let it top and tail your logs without the need to perform the nightly HUP. So no dropped connections and no missing log transactions if your logging bytes served.
The doc's say
The rotation occurs at the beginning of this interval. For example, if the rotation time is 3600, the log file will be rotated at the beginning of every hour; if the rotation time is 86400, the log file will be rotated every night at midnight.
If you look inside rotatelogs it's not obvious how they're doing this as there's no magic numbers at all until you spot the line:
tLogStart = (now / tRotation) * tRotation;
And this will align the start time with the start of the current hour (for tRotation == 3600) or the previous midnight (for tRotation == 86400). Who knows what it'll do for any other numbers, e.g. 16432?
Edit: I forgot to say that we've seen problems with graceful not behaving very gracefully. Specifically, just leaving child processes hanging. This seems to have been acknowledged as v2.2.12 now has a new config directive GracefulShutdownTimeout which specifies a timeout after which the httpd process terminates irrespective of any requests being served.
HTH
'Avahappy,
Why is loosing one line such a big deal? If its that important, perhaps the logs need to be printed to green bar as a backup.
Yep, as per the other answers, the issue is that apache will continue to write to the file handle it has open.
We (and lots of other people) use cronlog to avoid this. In this setup apache pipes the logs to cronolog, and the server does not need to be restarted/reloaded.
