84

I have a constantly running script that I output to a log file:

script.sh >> /var/log/logfile

I'd like to add a timestamp before each line that is appended to the log. Like:

Sat Sep 10 21:33:06 UTC 2011 The server has booted up.  Hmmph.

Is there any jujitsu I can use?

13 Answers 13

80

You can pipe the script's output through a loop that prefixes the current date and time:

./script.sh | while IFS= read -r line; do echo "$(date) $line"; done >>/var/log/logfile

If you'll be using this a lot, it's easy to make a bash function to handle the loop:

adddate() {
    while IFS= read -r line; do
        echo "$(date) $line"
    done
}

./thisscript.sh | adddate >>/var/log/logfile
./thatscript.sh | adddate >>/var/log/logfile
./theotherscript.sh | adddate >>/var/log/logfile
  • 3
    @Nils it's a trick to prevent read from trimming whitespace at the beginning and and of the line. It sets IFS (bash's Internal Field Separator, basically a list of whitespace characters) to empty for the read command. – Gordon Davisson Sep 15 '11 at 21:43
  • 2
    ...and -r ignores the escape-character "\". This should really work in all cases - great pice of scripting. – Nils Sep 16 '11 at 20:47
  • 7
    @Nils it's not completely bulletproof, since some implementations of echo interpret escape sequences. If you really want it not to mess with the content (other than adding dates), replace the echo command with printf "%s %s\n" "$(date)" "$line" – Gordon Davisson Sep 17 '11 at 2:31
  • 4
    You might be interested on a ISO-8601 compliant date/timestamp: date -u +"%Y-%m-%dT%H:%M:%SZ" or maybe more pretty date +"%Y-%m-%d %T". – Pablo Bianchi Jul 29 '17 at 1:40
  • 1
    while this script works as expected, it spawns a new process (execute date) for each log line, which can be a massive drawback depending on your machine and the amount of logs. I would rather suggest to use ts if available, see the answer from @willem – Michael Schaefers May 9 '18 at 13:20
46

See ts from the Ubuntu moreutils package:

command | ts

Or, if $command does automatic buffering (requires expect-dev package):

unbuffer command | ts
14

The date command will provide that information

date -u
Sat Sep 10 22:39:24 UTC 2011

so you can

echo $(date -u) "Some message or other"

is that what you wanted ?

  • Using the date command was kind of what I had in mind, but I can't really add that to the script itself, so what I'm looking for is a way to change this line: "script.sh >> /var/log/logfile" to append the date. – Antonius Bloch Sep 10 '11 at 21:53
  • In that case redirect the output of your script to a named pipe and have a daemon listening for output which takes the script output and adds a date before writing it to a log file. You can probably modify the script I wrote here to do this. I would do it because it interests me but it's late in the UK and I have an early start tomorrow. – Iain Sep 10 '11 at 22:12
12

You can simply echo the command outputs to the logfile. ie,

echo "`date -u` `./script.sh`" >> /var/log/logfile

It really works :)

Example:

[sparx@E1]$ ./script.sh 
Hello Worldy
[sparx@E1]$ echo "`date -u` `./script.sh`" >> logfile.txt
[sparx@E1]$ cat logfile.txt 
Mon Sep 12 20:18:28 UTC 2011 Hello Worldy
[sparx@E1]$ 
  • Hmm isn't working for me. – Antonius Bloch Sep 12 '11 at 19:11
  • What are you getting when you execute the command ? – SparX Sep 12 '11 at 20:15
  • 6
    That puts a timestamp before the entire output of ''./script.sh'', not before each line. – clacke Mar 9 '15 at 15:01
8

Make a config.sh file

#!/usr/bin/env bash
LOGFILE="/path/to/log.log"
TIMESTAMP=`date "+%Y-%m-%d %H:%M:%S"`

When you need to send to log file use

#!/usr/bin/env bash
source /path/to/config.sh

echo "$TIMESTAMP Say what you are doing" >> $LOGFILE

do_what_you_want >> $LOGFILE

Log file will looks like

2013-02-03 18:22:30 Say what you are doing

So it will be easy to sort by date

  • 6
    Your ''config.sh'' will run ''date'' exactly once, on ''source .../config.sh''. – clacke Mar 9 '15 at 15:03
6

You mean like:

(date && script.sh) >> /var/log/logfile
  • My god, people, everyone is doing back tick replacement, named pipes, etc. Just enclose both the date command and the script in parens! The guy who has the function has a legit case if there's multi-line output and the log needs to look pretty with the date on every line, but most of these solutions are overkill that doesn't use shell semantics. – cjc Sep 12 '11 at 21:11
  • 6
    That will only add the timestamp once per execution of script.sh. The OP needs a timestamp per line. – Dave Forgac Oct 10 '12 at 14:10
  • 1
    although this doesn't answer the OP question I still found it useful info. – User Nov 2 '12 at 20:14
3

Try this

timestamp()
{
 date +"%Y-%m-%d %T"
}

Call this timestamp function in every echo command:

echo "$(timestamp): write your log here" >> /var/log/<logfile>.log
  • @ shazbot:Thanks for editing, that was a typo error, i didnt notice. – Sanjay Yadav Jun 15 '15 at 7:08
3

The accepted answer https://serverfault.com/a/310104 can be a bit slow, if a lot of lines have to be processed, with the overhead of starting the date process allowing about 50 lines per second in Ubuntu, and only about 10-20 in Cygwin.

When bash can be assumed a faster alternative would be the printf builtin with its %(...)T format specifier. Compare

>> while true; do date; done | uniq -c
     47 Wed Nov  9 23:17:18 STD 2016
     56 Wed Nov  9 23:17:19 STD 2016
     55 Wed Nov  9 23:17:20 STD 2016
     51 Wed Nov  9 23:17:21 STD 2016
     50 Wed Nov  9 23:17:22 STD 2016

>> while true; do printf '%(%F %T)T\n'; done | uniq -c
  20300 2016-11-09 23:17:56
  31767 2016-11-09 23:17:57
  32109 2016-11-09 23:17:58
  31036 2016-11-09 23:17:59
  30714 2016-11-09 23:18:00
1

The below is my log file contents

xiongyu@ubuntu:~/search_start_sh$ tail restart_scrape.log 

2017-08-25 21:10:09 scrape_yy_news_main.py got down, now I will restart it

2017-08-25 21:10:09 check_yy_news_warn.py got down, now I will restart it

2017-08-25 21:14:53 scrape_yy_news_main.py got down, now I will restart it

some of my shell contents is as below

log_file="restart_scrape.log"
TIMESTAMP=`date "+%Y-%m-%d %H:%M:%S"`
echo "$TIMESTAMP $search_py_file got down, now I will restart it" | tee -a $log_file 
1

Another option is to setup a function to call each time you want to output data in your code:

PrintLog(){
  information=$1
  logFile=$2
  echo "$(date +'%Y-%m-%d %H:%M:%S" $information} >> $logFile
}

Then every time in your code you want to send it to the log file call

PrintLog "Stuff you want to add..." ${LogFileVariable}

Easy peasy....

0

Pipe a "sed":

script.sh | sed "s|^|$('date') :: |" >> /var/log/logfile
  • as for the other answer, this is just bash running date once. sed is not updating the time per line. – Evan Benn Nov 8 '18 at 22:35
0

You might note that awk runs fast,

gawk '{ print strftime("[%Y-%m-%d %H:%M:%S]"), $0 }'

yes |head -5000000 |gawk '{ print strftime("[%Y-%m-%d %H:%M:%S]"), $0 }' |uniq -c
 461592 [2017-02-28 19:46:44] y
 488555 [2017-02-28 19:46:45] y
 491205 [2017-02-28 19:46:46] y
 498568 [2017-02-28 19:46:47] y
 502605 [2017-02-28 19:46:48] y
 494048 [2017-02-28 19:46:49] y
 493299 [2017-02-28 19:46:50] y
 498005 [2017-02-28 19:46:51] y
 502916 [2017-02-28 19:46:52] y
 495550 [2017-02-28 19:46:53] y
  73657 [2017-02-28 19:46:54] y

But, sed runs much faster,

sed -e "s/^/$(date -R) /"

yes |head -5000000 |sed -e "s/^/$(date -R) /" |uniq -c
5000000 Tue, 28 Feb 2017 19:57:00 -0500 y

However, on closer inspection, set does not seem to change time,

vmstat 1 | sed -e "s/^/$(date -R) /"
  • this is because it is bash evaluating "s/^/$(date -R) /" and running date once before sed. Sed is passed a static string. – Evan Benn Nov 8 '18 at 22:34
0

This script print the output in terminal and also saves in log file.

#!/bin/bash

MY_LOG=/var/log/output.log

echolog(){
    if [ $# -eq 0 ]
    then cat - | while read -r message
        do
                echo "$(date +"[%F %T %Z] -") $message" | tee -a $MY_LOG
            done
    else
        echo -n "$(date +'[%F %T %Z]') - " | tee -a $MY_LOG
        echo $* | tee -a $MY_LOG
    fi
}

echolog "My script is starting"
whoami | echolog

Sample output:

[2017-10-29 19:46:36 UTC] - My script is starting
[2017-10-29 19:46:36 UTC] - root

Your Answer

By clicking "Post Your Answer", you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.