How to grep httpd error_log within a time range?

Question

How can I grep only lines from a huge (120GB) httpd error_log based on a time range, say:

from 2011-11-15 11:30 pm
to   2011-11-16 01:30 am

Thanks!

Answer 1

You'll probably have to do some drill down, I'd start by getting the date range:

grep -e "2011\-11\-[15-16] " error_log > filtered
grep -v -e "2011\-11\-15 [0-10]:" | grep -v -e "2011\-11\-15 11:[0-29]" > filtered
grep -v -e "2011\-11\-16 [2-23]:" | grep -v -e "2011\-11\-16 01:[31-59]" > filtered

cat filtered

The most efficient way I can think of but haven't done is to find the start and end bytes of your date range and get that; (which is apparently possible with grep) but I dont know how to get a range of bytes from a file - probably takes some awk skills

Edit: Since this was an interesting question - I did some more digging:

You can get the first byte offset by doing:

 # Get first byte offset, leftmost number is the offset...
grep -m 1 -b "2011-11-15 11:3" error_log
 # Get last byte offset
grep -m 1 -b "2011-11-16 01:3" error_log

 #(Subtract first number from last number to get byte length) Then do:

dd if=error_log of=filtered bs=c skip=<first number> count=<last_byte#-first_byte#>