Logging to a file, prepending a timestamp to each line (Debian)

Question

I am trying to log output from some shell scripts of mine to a file, and I want the timestamp prepended to each line. From what I understood, that's what logger(1) is for, but I don't want the output to go to /var/log/messages, and I can't see that this is possible using logger. I'm using Debian by the way.

What is the best way to do this?

—Oliver

Answer 1

logger's function is actually to shunt messages to syslog; default configs will prepend a timestamp and write the logs out to /var/log/messages, but that doesn't mean that logger's purpose is to prepend a timestamp.

One way to handle this would be to modify your syslog configs such that your messages routed via logger go to a special file - use the "-p" flag to logger to stipulate a facility.priority pair (perhaps one of the user[1..7] facilities), and configure your syslogd to log that facility to a special file.

Alternatively, you could whip up a quick shell function to simply prepend the timestamp:

Bodacious:~ james$ timestamp () {
> f=`date`
> echo $f $*
> }
You have new mail in /var/mail/james
Bodacious:~ james$ timestamp a line of logs
Tue 18 Jan 2011 22:40:24 EST a line of logs
Bodacious:~ james$ 

On my system, this is going to result in the shell forking /bin/date once per line of output. This is inefficient, but probably acceptable at small scales.

Answer 2

I needed something like that, and google got me here. Using something like syslog is definitely an overkill for just trying out to time something, and the printout-every-second is too inconvenient.

Here's a one-liner that seems to work fine:

cmd | { while read R; do echo "$(date +%s) $R"; done }

and a variant that prints the time it took to produce each line:

cmd | { T=$(date +%s); while read R; do T2=$(date +%s); echo "$((T2-T)) $R"; T=$T2; done }

(This is using seconds, probably not too hard to throw in %N in there, but at that level a shell solution is probably not the right tool anyway.)

Answer 3

You could use -t to tag each line, and then get your syslog server to filter to a different file based on that tag.

(Edited to take out incorrect reference to -f flag)

Answer 4

Using the answer from Command to prepend string to each line?, I found this the easiest solution:

cmd | sed "s/^/`date +'%a %b %d %k:%M:%S'` /"