2

I run an apt-cacher-ng server on Ubuntu Linux which writes logs in the following format:

1299745593|O|149406|XXX.XXX.XXX.XXX|uburep/pool/main/t/tiff/libtiff4_3.9.2-2ubuntu0.4_amd64.deb
1299745593|O|10154976|XXX.XXX.XXX.XXX|uburep/pool/main/l/linux-firmware/linux-firmware_1.34.4_all.deb     
1299748529|O|39368|XXX.XXX.XXX.XXX|uburep/pool/main/n/nagios-nrpe/nagios-nrpe-server_2.12-4ubuntu1_amd64.deb 
1300155440|O|680100|XXX.XXX.XXX.XXX|uburep/pool/main/t/tzdata/tzdata_2011c-0ubuntu0.10.04_all.deb

It shows the timestamp, direction (in or out), byte count, IP and filename.

Every time a line is written to it, I'd like to also send that line to another program. I will have this program insert the line into a database so that I can crunch some statistics about how much bandwidth we're saving through operating a caching server.

I do not want to cat the log file every X minutes (via cron) looking for new entries as it'd be somewhat computationally uneconomical. Instead I'd prefer to have a daemon monitor the log, and when a change is detected, each line is sent to my database-insertion script.

Will swatch achieve this, or are there better options?

3 Answers 3

1

Depending on how daemon-y you want to get, you might get away with something like:

tail -f $FILE | xargs -n 1 frobwidget.sh
2
  • That was my initial line of thinking, but I'd like it to be "more daemony-y" if that makes sense.
    – mlambie
    Mar 15, 2011 at 7:16
  • In that case I'd suggest getting apt-cacher-ng to log to syslog, if it supports it, then use syslog to dump it in the DB or pipe it to your programme. If it doesn't support syslog, see if your syslogd supports listening on extra sockets (with custom log formats) and get it to replace the file with a socket.
    – user18176
    Mar 15, 2011 at 8:43
0

If the logs are being sent through syslog then you can add a syslog rule to send them elsewhere already. How you go about it will depend which flavour of syslog you're using (ksyslogd, syslog-ng, rsyslog, etc).

If not, swatch looks like a good solution to the problem.

Probably the only other way I could think of doing this without dodgily hacking things up too badly is to write your script to open a named pipe in the place of the log files location - and to write the logs out to the DB and an alternate location on disk - but this would require the named pipe to be in place before the apt-cacher-ng daemon started.

0

Have you looked at using syslog-ng to log to mysql? Using a pipe destination in syslog-ng sounds very similar to what you are trying to do.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .