I come across quite a few use cases where it would be very useful to take input from a (usually newline-delimited) stream and summarise it in a top-like fashion (see top, iotop, etc). A kind of on-the-fly pivot table.

e.g. Take the log-esque input:

I heard A from unit 1 and it said "Great!" 56
I heard A from unit 2 and it said "Oh no!" 42
I heard C from unit 1 and it said "Waiting for input." 33
I heard B from unit 3 and it said "Stopped." -1

From this, we could run a tool with a regex and group indicators:

topify [lineout] [regex] [name #1] [group #1] [name #2] [group #2] [All other columns name position]
         lineout is the number of lines before removing it from the display
         regex is a regex of the lines to match, complete with group indicators
         name #n is a string for the title of column n
         group #n is the number of the group in the regex


topify '/^I heard ([A-Z]) from unit ([1-9]) and it said "(.*)" ([-0-9]*)$/' Unit 2 Status 1 Message 3 RetVal 4

This would display in an interactive way, such that the columns could be selected/reordered, etc.:

Unit Status Message            Retval
1    C      Waiting for input. 33
2    A      Oh no!             42
3    B      Stopped.           -1

I understand the brittleness of it, but I'd be really surprised if it hadn't been built before and wanted to check before I go into building it. I also appreciate it isn't particularly complicated to write so maybe everyone has just implemented their own solution...

Has anyone seen a tool like this?

(Please excuse the tags I've used here. I know I may be pushing/breaking the rules of some tags, but this is very generalised. Suggestions welcome.)

  • This really is something that you ought to write yourself… the closest thing I can think of is the watch command.
    – MikeyB
    Aug 20, 2013 at 1:11
  • That was my first thought, too, but watch doesn't group things together or filter with smarts. I thought about using an awk/watch/shell combination but it's one step more complex than that. Aug 20, 2013 at 1:28
  • I think that awk is the way to go... using a keyed array.
    – Floris
    Aug 20, 2013 at 4:01

1 Answer 1


You don't need to write a tool, the standard unix toolset can accommodate you just fine.

echo -e 'Unit\tStatus\tMessage\t\t\tRetval'
cat /var/log/filename | awk '{match($0,"\".*\"",a)}{print $6 "\t" $3 "\t" a[0] "\t\t" $NF}' |sort -k<fieldnum>

Put that in a .sh file and run watch on it.

  • 1
    I think you just wrote a tool. Sep 11, 2013 at 2:04
  • Good effort, but I think you missed a few criteria "(see top/iotop, etc.)...on-the-fly....in an interactive way such that the columns could be selected/reordered..." It's possible that you could rewrite the command, but this isn't very flexible once you start the script and you could miss information by doing so. Sep 11, 2013 at 5:09
  • Fair enough, it doesn't match criteria, but then not much other than specifically designed custom software will, if you know want to write it yourself look at the ncurses development libraries, other than that i'm fairly certain you'll find nothing to meet your exact requirements.
    – Aaron Tate
    Sep 11, 2013 at 6:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.