Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm looking for a cronolog-like tool that will keep only last n lines or last x minutes of logs piped to it and discard everything else

Is there such a beast?

UPDATE:

I know about logrotate and it renames and zips old logfiles, which is not what I want.

I want to discard old log lines and keep only recent lines.

Like i.e. doing this every so often: tail -10000 logfile > logfile.new mv logfile.new logfile except that with this technique you will most certainly lose log lines and you have to restart or otherwise signal the logging application to reopen the logfile.

share|improve this question

5 Answers 5

up vote 4 down vote accepted

Logrotate can be made to only keep one copy of a logfile... If you RTFM you'll find the following bit regarding configuration settings:

rotate count
    Log files are rotated count times before being removed or mailed to the 
    address specified in a mail directive. If count is  0,  old  versions  
    are  removed  rather than rotated.

You can couple rotate with size, again from the logrotate(8) man page, to keep the file size small. While not by number of lines but by k, M, G size.

size size
    Log  files  are rotated when they grow bigger than size bytes. If size is 
    followed by M, the size if assumed to be in megabytes.  If the G suffix is 
    used, the size  is  in gigabytes.   If  the k is used, the size is in 
    kilobytes. So size 100, size 100k, and size 100M are all valid.
share|improve this answer
    
thanks for rtfming for me ;) –  Aleksandar Ivanisevic Jul 20 '09 at 17:34

You can use logrotate and put

tail -10000 logfile.0 > logfile.0.new 
mv logfile.0.new > logfile.0

as part of postrotate command. logrotate allows you to specify postrotate commands.

share|improve this answer
    
Not a bad idea, but this keeps two logfiles, one current and one old. And it doesn't keep the fixed amount of logs. –  Aleksandar Ivanisevic Jul 20 '09 at 11:46

You will have to restart or signal the application anyway. The application somehow has to get to know the new offset for seek()ing or hast to reopen the filehandle when you trim the logfile.

share|improve this answer
    
ideally the same app would both log and trim, think about a replacement for logwatch. I would like to avoid sending any signals to the original app, if at all possible. –  Aleksandar Ivanisevic Jul 19 '09 at 11:13

Not entirely what you're saying, but you might check out logrotate(8). From the man page:

logrotate is designed to ease administration of systems that generate large numbers of log files. It allows automatic rotation, compression, removal, and mailing of log files. Each log file may be handled ... when it grows too large.

It's installed by default on RHEL and derivitives. I don't know about anything else like Ubuntu/derivitives or Windows.

share|improve this answer
    
I know about logrotate and it renames and zips old logfiles, which is not what I want. I want to discard old log lines and keep only recent. Like i.e. doing this every so often: tail -10000 logfile > logfile.new mv logfile.new logfile except that with this technique you will most certainly lose log lines and you have to restart or otherwise signal the logging application to reopen the logfile –  Aleksandar Ivanisevic Jul 18 '09 at 16:05

At the end I've solved it like this (not the most elegant, but it works):

in apache (or whoever logs):

CustomLog "|/usr/local/cronolog/sbin/cronolog /var/tmp/mylog.%Y%m%d.log" logformat

in cron.daily:

find /var/tmp/ -name mylog* -mtime +$days | xargs --no-run-if-empty rm

this will delete old logs

and finally in the analyzing script:

lastdate=$(date -d "$INTERVAL sec ago" +%Y-%m-%dT%H:%M:%S)

grep -h $SEARCH /var/tmp/mylog* | awk -v lastdate="$lastdate" '$1>lastdate { print }' > /tmp/cutlog

and then work with /tmp/cutlog

the above example assumes ISO timestamps in the first field like: 2009-07-20T13:52:32

not the most elegant way but it does what I want. Maybe one day I'll write a feature for cronolog that would do the same thing :)

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.