Apache vhost log rotate by date

Question

I've got Apache 2.2 running on Debian 7 (on a VPS), with virtual hosts defined for several different websites.

I've set up individual error and access logs for each vhost, but I've not found a way to rotate/archive the logs to my liking.

Of course the default Apache logs on Debian are rotated by logrotate, but I don't find this particularly satisfactory for various reasons - primarily because I want archived logs to be named by date, and because configuring it separately for each vhost seems cumbersome. I'm also unsure about deleting old logs automatically; I might want to do this manually only after downlading them from the server.

My ideal solution would do the following every month (or maybe week) :

• Append the date to the filename, or move the file to a date-named subfolder.
• Gzip the archived files.

I also need this to be easy to set up for multiple vhosts.

Ideally, it should not involve restarting apache (though a graceful restart is not the end of the world).

What is the best way to set this up? I'm sure it's been done before...

Answer 1

(Do I detect a litlle bit of NIH?)

Here's why you shouldn't do that all by yourself:

1. It's better to extend than to reinvent: In essential, what you want to do is already covered with logrotate. If you don't like the default behaviour of it, you can thankfully configure it to behave differently, via it's pre/post-actions, naming conventions etc.

2. Solving an already solved problem: The guys that have contributed to logrotate all have encountered your problem before, and then some. At some point, you most likely run into the same detail problems they faced and solved with way more experience and feedback...do you really want to waste your energy to solve them again?

3. Not deleting old logs is a very bad idea. This results in a waiting game for filling up your log partition – depending on your setup, your applications, or even your whole server, will come to a screeching halt. Preferably at the height of your day-to-day business.

   If you need to keep old logs around for accounting reasons (security, financial, gov), you have to implement an archiving solution for them.

4. Supporting homebrew: If you ever run into problems, there's no one else to ask, since only you use this solution - whereas logrotate is used by many others.

5. Setting up is easy: Just cp and sed your vhost configurations accordingly. Strive for automation.

Note that above reasons are not cast in stone, and do not apply everywhere. It's not a bad idea to challenge existing solutions from time to time (there might be a better solution), but in this case, I would consider this an exercise in futility.

Answer 2

I found various solutions to this problem on the web, none of them totally satisfactory but several of them acceptable. These included Apache's own rotatelogs script, which involves piping the log to the script along with arguments which tell the script how to split and name the files.

In the end, however, I did end up using logrotate. It turns out, I am ashamed to say, that the online manpage I had been reading was outdated, and in fact logrotate now has the ability to name files by date, which was the main thing I wanted.

I was also able to use wildcards to avoid having to set up a new conf file for logrotate every time I add a vhost in the future.

For anyone else trying to do this, the logrotate config I used was the following:

/var/www/*/logs/*.log {
    weekly
    missingok
    rotate 52
    compress
    delaycompress
    dateext
    dateformat .%Y%m%d
    extension .log
    olddir old
    create 640 root root
    sharedscripts
    postrotate
        /etc/init.d/apache2 reload > /dev/null
    endscript
    prerotate
        if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
            run-parts /etc/logrotate.d/httpd-prerotate;
        fi; \
    endscript
}

I'm sure I'll change it in the future, but as a starting point it's fine.

(Note, the olddir is necessary because otherwise *.log also matches last week's log file...)

Answer 3

Another alternative could be doing it via CRON. Here is how I do;

#!/bin/bash

for file in /var/www/*/logs/*.log; do
    # zip and truncate file (dont remove cos apache cannot write on it unless restarted)
    zip $(dirname $file)/$(date +"%Y-%m-%d")-$(basename $file).zip $file && truncate -s 0 $file
done

This script will produce a zipped log file like /var/www/example.com/logs/2010-01-01-access.log.zip.