I am trying to count the number of SQL queries per second from a log file and I want to do it in real time by pipeing stdout from grep into some command. ( I am doing some performance testing )

I could write it myself, but thought for sure this would exist.

I looked at wc but didn't see an option to allow this.

I could also use it to count requests per second by piping a tail from the access log.


6 Answers 6


pv is your command! Pipe Viewer prints stats about the data passing through it, and can run anywhere in your pipeline, since it pipes stdin directly over to stdout. For example:

tail -f /var/log/nginx/access.log | pv --line-mode --rate > /dev/null

The pv command prints to stderr the current number of lines per second (the default is bytes per second), which, for this particular data source (Nginx's default log file), equates to incoming web requests per second. I only care about the counts, so I pipe stdout into /dev/null. There are also options like:

  • -b (total number of lines),
  • --average-rate (average rate since starting), and
  • --timer (tracks how long the pipe has been going).

If you don't say --line-mode, it'll count bytes, which is probably not what you want for server logs, but could be handy elsewhere.

Final note: ... | pv -lb > file.txt is a lot like ... | tee file.txt | awk '{printf "\r%lu", NR}', which is also handy for counting lines, but the pv call is way shorter, though the output is not quite as exciting — pv updates every second by default, while that awk command updates continuously.

  • Is it possible to get only one value from pv? I need average value for time period for monitoring purpose. So i need to assign value to variable.
    – Sonique
    Apr 11, 2016 at 4:10
  • @Sonique it's a kludge, since that's not what pv is built for (I'd reach for awk if I were you), but of course it's possible. Suppose twilight stream --timeout 5 is a command that will sample from the Twitter spritzer for 5 seconds and then quit: RATE="$(twilight stream --timeout 5 | pv --line-mode --rate --force 2>&1 1>/dev/null | tr -s '\r\n' '\n' | tail -1)", then echo $RATE produces something like "[40.8 /s]" (note the additional --force flag, since pv's stderr is no longer a TTY).
    – chbrown
    Apr 12, 2016 at 3:47
  • Doesn't seem to work reliably for me for all programs; removing the > /dev/null shows that the output is now "block chunked" and no longer streams smoothly. Perhaps unbuffer is needed on some producing programs so that they don't switch to block output buffering when they detect that they're being piped?
    – nh2
    Nov 1, 2018 at 12:18

Maybe you should try logtop ?

tail -f foobar.log |logtop
watch -n 3 "wc -l logfile"

man page

watch - execute a program periodically, showing output fullscreen By default, the program is run every 2 seconds; use -n or --interval to specify a different interval.

  • Good reminder about watch, but this approach doesn't really count queries (lines written) per second as requested. Instead, it counts the total lines present in the log file every second.
    – rinogo
    Sep 25, 2020 at 16:04
watch -n 5 "mysqladmin status | awk -F'  ' '{ print \$NF }'"
  • I couldn't get this to work. You sure about the syntax?
    – digidigo
    Jan 31, 2013 at 17:04
  • Which error do you get? Have you configured ~/.my.cnf to run mysqladmin without prompting for password?
    – quanta
    Feb 1, 2013 at 0:57

If you're on a system where you can't install pv or logtop suggested above, and the log file is too large to read the whole thing with wc, you can tail-watch it instead:

MYLOGS=$(timeout 10 tail -f /path/to/log.log)
expr $(echo "$MYLOGS" | wc -l) / 10

Unfortunately, some of the tools given in these answers are not installed by default in most of the Linux distos.

Here is a piece of command line that should work everytime (Perl is almost everywhere)

tail -f /var/log/some_logfile.log | perl -e 'while (<>) {$l++;if (time > $e) {$e=time;print "$l\n";$l=0}}'

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .