How can I keep my web server's log files from becoming too large?

Question

Note: I am using Apache2 (on Linux), but I asked in a general sense (for Linux only) because I'd also like to know the "best" way to do accomplish this in a general (since I'm about to deploy a large site on Nginx, or Cherokee).

My log files are becoming huge after just a few weeks. I need to keep them around temporarily, but I'd like to delete entries that are older than 2 weeks, or so. Is this possible, and how do I do it?

Answer 1

You could use logrotate. It rotates logs according to a configuration file for a specific service. It is usually run by cron on a daily basis.

An example of a config file for apache at /etc/logrotate.d/apache2

/var/log/apache2/*_log {
 daily
 rotate 31
 missingok
 compress
 delaycompress
 sharedscripts
 postrotate
  if [ -f "`. /etc/apache2/envvars ; echo ${APACHE_PID_FILE:-/var/run/apache2.pid}`" ]; then
     /etc/init.d/apache2 reload > /dev/null
  fi
 endscript
 }

This would:

• rotate logs everyday
• keep 31 rotated log files
• Compress rotated logs but keep latest rotated one uncompressed (delaycompress)
• Reload the process

If you dont want the process to be reloaded, then you should use copytruncate, which will copy the current content onto a new file, compress it and then truncate current logfile. In this case you dont need sharedscripts, postrotate and endscript.

Answer 2

The canonical tool for handling logs is logrotate.