The accepted answer https://serverfault.com/a/310104 can be a bit slow, if a lot of lines have to be processed, with the overhead of starting the date
process allowing about 50 lines per second in Ubuntu, and only about 10-20 in Cygwin.
When bash
can be assumed a faster alternative would be the printf
builtin with its %(...)T
format specifier. Compare
>> while true; do date; done | uniq -c
47 Wed Nov 9 23:17:18 STD 2016
56 Wed Nov 9 23:17:19 STD 2016
55 Wed Nov 9 23:17:20 STD 2016
51 Wed Nov 9 23:17:21 STD 2016
50 Wed Nov 9 23:17:22 STD 2016
>> while true; do printf '%(%F %T)T\n'; done | uniq -c
20300 2016-11-09 23:17:56
31767 2016-11-09 23:17:57
32109 2016-11-09 23:17:58
31036 2016-11-09 23:17:59
30714 2016-11-09 23:18:00
As pointed out after the original answer (see below), an awk-based solution is the most efficient, when really high throughput is needed, and it has the advantage of not being bash-specific syntax. (For instance, in zsh
, the %(..)T
format does not work.)
Remark. On my current OpenSuse work PC (July 2021) performance is significantly up, doing 1200 lines per second with date
and 265,000 lines per second with printf
. Not entirely clear, how this particular performance jumped by almost two orders of magnitude compared to my laptop from 5 years ago.
Update. Just adding new benchmarks (still same OpenSuse PC, May 2023), and adding awk
solution, as suggested by StéphaneGourichon in the comments.
>>> yes | while IFS= read -r line; do echo "$(date) ${line}"; done | uniq -c
561 Tue 09 May 2023 10:20:39 AM CEST y
704 Tue 09 May 2023 10:20:40 AM CEST y
695 Tue 09 May 2023 10:20:41 AM CEST y
696 Tue 09 May 2023 10:20:42 AM CEST y
>>> yes | while IFS= read -r line; do printf "%(%F %T)T: %s\n" -1 "${line}"; done | uniq -c
6935 2023-05-09 10:23:21: y
22962 2023-05-09 10:23:22: y
20426 2023-05-09 10:23:23: y
18212 2023-05-09 10:23:24: y
18406 2023-05-09 10:23:25: y
>>> yes | awk '{print strftime("%F %T"), $0}' | uniq -c
520932 2023-05-09 10:24:51 y
587067 2023-05-09 10:24:52 y
586600 2023-05-09 10:24:53 y
587226 2023-05-09 10:24:54 y
579186 2023-05-09 10:24:55 y
572274 2023-05-09 10:24:56 y
Like the while-loop based variants, one date is added per line, and within-line whitespace is preserved unmodified.
>>> yes | sed 's/y/This is a longer text/' | awk '{print strftime("%F %T"), $0}' | uniq -c
331324 2023-05-09 10:28:09 This is a longer text
504857 2023-05-09 10:28:10 This is a longer text
script.sh | gawk '{ print strftime("[%Y-%m-%d %H:%M:%S]"), $0 }' >> /var/log/logfile