Advice on apache log files

Question

I have a small VPS that a friend and I use to host a few websites. We are both relatively inexperienced with server management and are unsure of an effective way to deal with the logs that Apache generates.

Each site generates it's own log file in a directory structure similar to the one below:

/var/www/{user]/{sitename}/logs

The problem is that some of these are getting quite large (300mb). I am considering installing logrotate to help deal with this size problem. However I am unsure of an effective way to deal with the logs since these are not critical sites and thus the logs in my opinion aren't that important.

What are recommended practices for working with logs? and what is your advice/opinions on such matters?

Answer 1

Logrotate is definetly the way to go. Since the logs are growing in size you can use logrotate to significantly compress the logs. Assuming your using apache modify the following

vi /etc/logrotate.d/httpd

You'll want it to look something like this:

"/var/log/httpd/site1/*.log" "/var/log/httpd/site2/*.log" {
weekly
rotate 7
compress
missingok
notifempty
sharedscripts
postrotate
/sbin/service httpd reload > /dev/null 2>/dev/null || true
endscript
}

• Weekly: Log files are rotated if the current weekday is less then the weekday of the last rotation or if more then a week has passed since the last rotation.
• Rotate 7: Log files are rotated 52 times before being removed or mailed to the address specified in a mail directive. If count is 0, old versions are removed rather then rotated.
• Compress: Old versions of log files are compressed with gzip to save disk space.
• Missingok: If the log file is missing, go on to the next one without issuing an error message.
• Notifempty: Do not rotate the log if it is empty
• Sharedscripts: Normally, prerotate and postrotate scripts are run for each log which is rotated, meaning that a single script may be run multiple times for log file entries which match multiple files. If sharedscript is specified, the scripts are only run once, no matter how many logs match the wildcarded pattern. However, if none of the logs in the pattern require rotating, the scripts will not be run at all.
• Postrotate /sbin/service httpd reload > /dev/null 2>/dev/null || true
• Endscript: The lines between postrotate and endscript (both of which must appear on lines by themselves) are executed after the log file is rotated. These directives may only appear inside a log file definition.

EDIT: As Lucas Mentioned: Another thing you might want to check out is why your logs are filling up. This can point you to some bad code, or perhaps someone trying to do something malicious, ie via forms etc

If you need more info on the options just use

man logrotate

Answer 2

I would indeed use logrotate, your 300 MB will turn into a mere 8 MB after compressing. You can also script it so you can backup the compressed logs to a remote server (or some other backup drive) every once in a while and then delete those logs.

If the sites are not critical you can discard the logs after a while (it's up to you to decide when you think they aren't useful anymore).

Another thing you might want to check out is why your logs are filling up. There might be some warnings in your code that can easily be fixed or are just unnecessary debug information.