Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I am trying to count the number of SQL queries per second from a log file and I want to do it in real time by pipeing stdout from grep into some command. ( I am doing some performance testing )

I could write it myself, but thought for sure this would exist.

I looked at wc but didn't see an option to allow this.

I could also use it to count requests per second by piping a tail from the access log.

share|improve this question
1  
    
I am looking for something more generally useful that what that question refers to. –  digidigo Jan 31 '13 at 17:05

3 Answers 3

up vote 11 down vote accepted

Maybe you should try logtop ?

tail -f foobar.log |logtop
share|improve this answer
    
Is this the log top you are referring to? github.com/JulienPalard/logtop –  digidigo Jan 31 '13 at 17:03
    
Yes, if you are using debian there is a package in repository packages.debian.org/wheezy/logtop –  klocek Jan 31 '13 at 21:45
    
dang I think this is what I need but I couldn't get it to compile on CentOs –  digidigo Feb 2 '13 at 4:07

pv is your command! Pipe Viewer prints stats about the data passing through it, and can run anywhere in your pipeline, since it pipes stdin directly over to stdout. For example:

tail -f /var/log/nginx/access.log | pv --line-mode --rate > /dev/null

The pv command prints to stderr the current number of lines per second (the default is bytes per second), which, for this particular data source (Nginx's default log file), equates to incoming web requests per second. I only care about the counts, so I pipe stdout into /dev/null. There are also options like:

  • -b (total number of lines),
  • --average-rate (average rate since starting), and
  • --timer (tracks how long the pipe has been going).

If you don't say --line-mode, it'll count bytes, which is probably not what you want for server logs, but could be handy elsewhere.

Final note: ... | pv -lb > file.txt is a lot like ... | tee file.txt | awk '{printf "\r%lu", NR}', which is also handy for counting lines, but the pv call is way shorter, though the output is not quite as exciting — pv updates every second by default, while that awk command updates continuously.

share|improve this answer
watch -n 5 "mysqladmin status | awk -F'  ' '{ print \$NF }'"
share|improve this answer
    
I couldn't get this to work. You sure about the syntax? –  digidigo Jan 31 '13 at 17:04
    
Which error do you get? Have you configured ~/.my.cnf to run mysqladmin without prompting for password? –  quanta Feb 1 '13 at 0:57

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.