Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I run an apt-cacher-ng server on Ubuntu Linux which writes logs in the following format:

1299745593|O|149406|XXX.XXX.XXX.XXX|uburep/pool/main/t/tiff/libtiff4_3.9.2-2ubuntu0.4_amd64.deb
1299745593|O|10154976|XXX.XXX.XXX.XXX|uburep/pool/main/l/linux-firmware/linux-firmware_1.34.4_all.deb     
1299748529|O|39368|XXX.XXX.XXX.XXX|uburep/pool/main/n/nagios-nrpe/nagios-nrpe-server_2.12-4ubuntu1_amd64.deb 
1300155440|O|680100|XXX.XXX.XXX.XXX|uburep/pool/main/t/tzdata/tzdata_2011c-0ubuntu0.10.04_all.deb

It shows the timestamp, direction (in or out), byte count, IP and filename.

Every time a line is written to it, I'd like to also send that line to another program. I will have this program insert the line into a database so that I can crunch some statistics about how much bandwidth we're saving through operating a caching server.

I do not want to cat the log file every X minutes (via cron) looking for new entries as it'd be somewhat computationally uneconomical. Instead I'd prefer to have a daemon monitor the log, and when a change is detected, each line is sent to my database-insertion script.

Will swatch achieve this, or are there better options?

share|improve this question
add comment

3 Answers

Depending on how daemon-y you want to get, you might get away with something like:

tail -f $FILE | xargs -n 1 frobwidget.sh
share|improve this answer
    
That was my initial line of thinking, but I'd like it to be "more daemony-y" if that makes sense. –  mlambie Mar 15 '11 at 7:16
    
In that case I'd suggest getting apt-cacher-ng to log to syslog, if it supports it, then use syslog to dump it in the DB or pipe it to your programme. If it doesn't support syslog, see if your syslogd supports listening on extra sockets (with custom log formats) and get it to replace the file with a socket. –  thsutton Mar 15 '11 at 8:43
add comment

If the logs are being sent through syslog then you can add a syslog rule to send them elsewhere already. How you go about it will depend which flavour of syslog you're using (ksyslogd, syslog-ng, rsyslog, etc).

If not, swatch looks like a good solution to the problem.

Probably the only other way I could think of doing this without dodgily hacking things up too badly is to write your script to open a named pipe in the place of the log files location - and to write the logs out to the DB and an alternate location on disk - but this would require the named pipe to be in place before the apt-cacher-ng daemon started.

share|improve this answer
add comment

Have you looked at using syslog-ng to log to mysql? Using a pipe destination in syslog-ng sounds very similar to what you are trying to do.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.