69
tail -f path

The above will output modifications to the file instantly, but I want to apply a filter to the output, only show when there is a keyword xxx in it.

How to approach this?

1
  • 1
    Marking answers as accepted helps others who read your questions and their answers to know which answer might be more likely to help them if they have a similar question or problem. You can revisit your earlier questions by clicking on your username. – Dennis Williamson Jul 5 '10 at 9:38
92

With Unix you can pipe the output of one program into another.

So to filter tail, you can use grep:

tail -f path | grep your-search-filter
3
  • But let's say you have 100 lines in the tail (end of the file), and these lines are the ones you want to filter. Your solution – TraderJoeChicago Nov 7 '11 at 15:12
  • 1
    If you only want to search the last 100 lines of a file try tail -100 path | grep xxx – AddersUK Nov 25 '11 at 23:19
  • I thought it wasn't working, until I realised grep was case-sensitive. grep -i will make it case insensitive. – Kit Johnson Jun 3 '20 at 23:41
24

Short answer: tail -f somefile | grep somepattern

However, this tends to fall short. Let's say you're tailing a file that gets rotated often (if its a debug log, it might be rotated multiple times). In that case tail -F is your friend. I'll let you look up the difference.

But tail -f and tail -F print out a bunch of lines first, which is often undesirable in this use-case, so in this case add -n0

tail -F -n0 somefile | grep somepattern

That will be fine, up until you want to do some other filtering, and then you need to beware of buffering. stdout is line-buffered by default when writing to a terminal but when fully-buffered when writing to a pipe. So the following will emit lines as soon as they are found, because tail is explicitly line-buffered (or it flushes its output at the end of each line), and grep is also line-buffered because its output is going to your terminal:

tail -F -n0 somefile | grep somepattern

But then you decide to use something like awk or cut to further process the output.

tail -F -n0 somefile | grep somepattern | awk '{print $3}'

And now you wonder where your output has gone... depending on the volume of logs, you may find that you do get output, but it will be a page at a time because now the stdout of grep is operating in fully-buffered fashion, and so awk receives it input 4kB at a time (by default).

In this case, you can tell grep to always make stdout line buffered by using the --line-buffered option.

tail -F -n0 somefile | grep --line-buffered somepattern | ...

However, most commands don't have an analogue of --line-buffered. In the case of more scriptable tools, you can use a function to flush the output (eg. in awk, the function is fflush(), which shares the same name as its C counterpart, tools like Perl and Python have something similar).

With the likes of cut you're likely out of luck; ... but you might try searching for unbuffer, which is I think something provided by the expect toolchain (I've never used it).

I hope you've found this useful.

Cheers, Cameron

2
  • 3
    I really appreciate the explanation as to why this works in line-buffered mode. That's an excellent insight, thank you. – Green Sep 11 '17 at 15:17
  • This should be marked as an answer! Thank you! You made my day! Actually tail -f file | grep pattern resulted in a 10-20 seconds delay, whereas the solution above made it out instantly. – Serhii Nadolynskyi Feb 11 at 14:31
2

and you can use multiple pipes and greps, and exclude things with grep -v, get case insensitivity with grep -i, etc.

i.e.: tail -100f /var/log/messages | grep -V ACPI | grep -i ata

start tailing 100 lines from the end, and keep tailing, first exclude any lines with ACPI, then show lines with ata, ATA, or any mix of those.

Another handy one is the ABC options, for the lines After, Before, and Context (lines before and after).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.