8

I have a set of 100 log files, compressed using gzip. I need to find all lines matching a given expression. I'd use grep, but of course, that's a bit of a nightmare because I'll have to unzip all files, one by one, grep them and delete the unzipped version, because they wouldn't all fit on my sevrer if they were all unzipped.

Anyone has a little trick on how to get that done quickly?

1
  • You mentioned the requirement of needing to find all lines matching a given expression. Does finding them mean that you need to know which file they're in? If not, more options are open to you, particularly those that pipe their output to the stdin of grep.
    – LarsH
    Oct 21, 2013 at 20:49

5 Answers 5

25

You might have a look at zgrep.

>$ zgrep -h
grep through gzip files
usage: zgrep [grep_options] pattern [files]
10

The zgrep program exists for this specific purpose.

http://linux.about.com/library/cmd/blcmdl1_zgrep.htm

4

Or, if your OS doesn't come with zgrep, something like this.

gunzip -c somefile1.gz [...] | grep 'string'
1
  • Which is how zgrep is actually implemented, as it is just a shell script (with some additional stuff to handle arguments and report errors properly).
    – liori
    Oct 21, 2013 at 20:57
3

There's a utility called zcat, which is a version of cat that works on gzipped files. In your situation, you could do something like the following:

zcat yourfile.gz | grep 'serverfault is awesome'
1
1

While zgrep appears to be the next best thing since sliced bread, I thought I'd post an alternative...

You can use piping to unzip the log files before grepping them, without taking up disk space. For example:

cat /var/log/program.log.*.gz | gunzip | grep 'stuff and things'
1
  • Damn... Apparently, I was a bit late to the party...
    – Soviero
    Oct 21, 2013 at 14:15

Not the answer you're looking for? Browse other questions tagged .