7

I'm using rotatelogs to create my daily apache logs in format host.<day>.<month>.<year>.access.log. Now I want to gzip and move log to different directory after it's been finished. How to do that?

Update: There was a little mistake. logrotate -> rotatelogs

  • Thanks for correcting the question, I've removed my inaccurate answer. – jscott Dec 23 '11 at 15:23
  • You can write a simple script that gzip the old files and move/copy them to a different location. This script can be invoked in a daily cron job. – Khaled Dec 23 '11 at 15:37
  • I'm not familiar with shell scripts. Specifically, I don't know how to generate filename string for yesterday's date. – Poma Dec 23 '11 at 15:39
6

You could use rotatelogs option -p to use a program to compress the log after the rotation. (See for reference: https://httpd.apache.org/docs/2.4/programs/rotatelogs.html)

-p program

If given, rotatelogs will execute the specified program every time a new log file is opened. The filename of the newly opened file is passed as the first argument to the program. If executing after a rotation, the old log file is passed as the second argument. rotatelogs does not wait for the specified program to terminate before continuing to operate, and will not log any error code returned on termination. The spawned program uses the same stdin, stdout, and stderr as rotatelogs itself, and also inherits the environment.

Example:

CustomLog "|bin/rotatelogs -p '/path/to/compress.sh' -l /var/log/logfile.%Y.%m.%d 86400"

compress.sh:

#!/bin/bash
file_to_compress="${2}"
compress_exit_code=0

if [[ "${file_to_compress}" ]]; then
    echo "Compressing ${file_to_compress} ..."
    tar --gzip --create --file "${file_to_compress}.tar.gz" "${file_to_compress}"

    compress_exit_code=${?}

    if [[ ${compress_exit_code} == 0 ]]; then
        echo "File ${file_to_compress} was compressed."
    else
        echo "Error compressing file ${file_to_compress} (tar exit code: ${compress_exit_code})."
    fi
fi

exit ${compress_exit_code}
  • IMHO this is the best answer (+1). 1) If you don't want to keep the uncompressed files, include --remove-files into tar. 2) Depending on your case, it might be also interesting to include --verify. – Mark Messa Aug 13 at 15:56
5

I've came up with the following script

#!/bin/sh
for file in $(ls /var/log/apache2/*.$(date +"%y.%m.%d" --date="1 day ago").access.log); do
    gzip $file
    mv $file.gz /var/log/apache2/archive
done;

And following cron entry

15 0    0 0 0   root    /mypath/myscript.sh
1

Logrotate will happily do the compressing for you. Just add:

compress

To the logrotate config for apache. There is also a neat option that delays the compressing by one day:

delaycompress

As for moving them, logrotate can't help you but a cron job like this can:

@daily mv /var/log/apache/*.gz /var/log/archive/
  • 2
    logrotate offers the olddir options which can move the logs -- see man logrotate. The OP edited the question to indicate he's using rotatelogs not logrotate. – jscott Dec 23 '11 at 17:46
  • Ah, that means I'm barking up the wrong tree. And I didn't even know about the olddir option. The shell script above seems like the best solution. – Ladadadada Dec 23 '11 at 18:01
  • 2
    logrotate is not rotatelogs! – MUY Belgium Apr 26 '13 at 9:45
1

I use find and crontab to accomplish this

# crontab -e

5 0 * * * /bin/find /path/to/logs/* -type f \( -mtime 1 ! -name "." ! -name ".gz" \) -print0 | xargs -0 gzip >/dev/null 2>&1;

-mtime n File’s data was last modified n*24 hours ago.

! -name "not ".*" hidden files

! -name "not ".gz" files which should already be compressed

-print0 & -0 to ensure spaces or special characters are piped correctly to xargs -> gzip (for filename safety)

find -exec could be used but has flaws where | xargs is more stable

1
CustomLog "|/opt/IHS/bin/rotatelogs -l /some/path/access_log.%Y.%m.%d 86400" common | gzip -9 /some/path/access_log.`date '+%Y.%m.%d'`
  • Hello arek and welcome on SF. Please take a moment to read this article. – Marco Oct 4 '17 at 19:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.