9

I need to search our mail logs for a specific e-mail address. We keep a current file named maillog as well as a week's worth of .bz2 files in the same folder. Currently, I'm running the following commands to search for the file:

grep [email protected] maillog
bzgrep [email protected] *.bz2

Is there a way combine the grep and bzgrep commands into a single output? That way, I could pipe the combined results to a single e-mail or a single file.

5 Answers 5

24

Another way is

{ grep ...; bzgrep ...;} >file

&& has the difficulty that the bzgrep wouldn't be run if the grep failed.

Note the mandatory space after the opening curly brace and semicolon after the last command. Alternatively, you can use the subshell syntax (parentheses instead of curly braces), which isn't as picky:

(grep ...; bzgrep ...) >file
3
  • +1 for the best general purpose shell approach to this issue. Shell grouping is a generally underutilized feature. Mar 8, 2011 at 23:07
  • 2
    Don't you mean () instead of {}? Dec 21, 2012 at 22:28
  • On arch linux this worked with () but not {}, either way +1 as this is what I needed! Jul 29, 2015 at 2:42
11

bzgrep automatically defaults to regular grep if a file isn't bzip-compressed. Thus the following should be sufficient:

bzgrep [email protected] maillog *bz2 | mail -s "logs yay" someuser@blah

oh also of course here's my obligatory GNU Parallel solution too:

parallel -m bzgrep [email protected] ::: maillog* *bz2 | mail -s "logs yay" someuser@blah

which could be a lot faster if you are checking a lot of files.

2

Here's another way to do it (assuming you're running bash, which you probably are):

cat <(bzgrep ...) <(grep ...)

Here bash is transparently feeding the output of the bzgrep and grep commands into cat as if they were files (and they sort of are under the hood, details in url at the bottom).

In your particular case I'd recommend Phil's solution, but the above is a good trick to keep in your bag.

If you're interested, you can read more here: http://www.tldp.org/LDP/abs/html/process-sub.html

1
  • Ah yes, the canonical answer for 'how do you diff the output of two processes'. A good trick to know. Mar 9, 2011 at 3:59
1

At the time of my writing this, the accepted answer's syntax was wrong for most, if not all, Bourne-derived shells, including bash. I suggested an edit to the top and accepted answer to fix that, but I was also inclined to add all this other information, and this would've been more of a rewrite instead of an edit.

You can use compound commands:

{ grep ...; bzgrep ...; } >file

..or subshells (note the parentheses instead of curly braces):

(grep ...; bzgrep ...) >file

..to group the commands. The subshell way has nicer syntax (more forgiving of lack of whitespace and allows you to omit the last semicolon), but it either forks a new process, or "pretends" to by having the commands run in an cleaned up environment. Both have advantages depending on what you want to do, which don't matter here, but are worth looking up if you want more proficiency with the shell.

Note: You can use pipelining with these tricks too, so you could do something like this:

{ grep ...; bzgrep ...; } | less

P.S. if you don't care about the ordering of the matches in your combined output, you could use a single & between the two commands, like so: { grep ... & bzgrep ...; }. Then the two commands run simultaneously: the grep gets launched and the shell puts it in the background, then the shell will run the bzgrep. (But there's a small caveat with that, with the explanation involving file redirection and file stream buffering potentially causing a very small portion of the lines in the output file to get split/mangled: whether you'd see this would depend on how your grep, bzgrep, and libc stdio.h functions are implemented. In most implementations, I believe piping the command before redirecting to a file will avoid the problem, so you could do { foo & bar; } | cat - >file as a workaround.)

0

You can tie the commands together with && which will allow you to run each command.

you could also add >> textfile.txt to the end of each command and have the output hit a file then mail that file out.

2
  • 2
    As geekosaur mentioned, && shouldn't be used here because grep's return value depends on whether it found something or not. If you did grep ... && bzgrep ..., if grep had no hits, it would return failure and the command would stop. >> is a good idea though, unlike >, it will add output to the end of an existing file.
    – DerfK
    Mar 8, 2011 at 22:36
  • right I had forgotten about the exit condition preventing the second command.
    – Mike
    Mar 8, 2011 at 22:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .