Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Hi all I am a bit new to grep etc.I did read the man page and searched internet before asking the question here.I am having a server which is running behind chain of reverse proxy servers.An application is giving me problem and I have to analyze the header response all those servers are accessible on SSH consoles so I decided to read Apache logs my problem is I want to have a script which can read the required part of logs from Apache and tell me exact thing.I have to do this on many places so the job becomes difficult. Here is a sample log I have

192.168.1.1 - - [18/Nov/2010:15:24:53 +0530] "GET /appl/pix/i/restore.gif HTTP/1.1" 304 187 "http://somedomain/appl/course/view.php?id=2" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; InfoPath.2)"
192.168.1.1 - - [18/Nov/2010:15:24:53 +0530] "GET /appl/pix/i/group.gif HTTP/1.1" 304 187 "http://somedomain/appl/course/view.php?id=2" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; InfoPath.2)"
192.168.1.1 - - [18/Nov/2010:15:24:53 +0530] "GET /appl/pix/i/return.gif HTTP/1.1" 304 188 "http://somedomain/appl/course/view.php?id=2" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; InfoPath.2)"
192.168.1.1 - - [18/Nov/2010:15:24:53 +0530] "GET /appl/pix/i/stats.gif HTTP/1.1" 304 187 "http://somedomain/appl/course/view.php?id=2" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; InfoPath.2)"
192.168.1.1 - - [18/Nov/2010:15:24:53 +0530] "GET /appl/pix/i/questions.gif HTTP/1.1" 304 187 "http://somedomain/appl/course/view.php?id=2" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; InfoPath.2)"
192.168.1.1 - - [18/Nov/2010:15:24:53 +0530] "GET /appl/pix/i/files.gif HTTP/1.1" 304 187 "http://somedomain/appl/course/view.php?id=2" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; InfoPath.2)"
192.168.1.1 - - [18/Nov/2010:15:24:53 +0530] "GET /appl/pix/i/user.gif HTTP/1.1" 304 187 "http://somedomain/appl/course/view.php?id=2" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; InfoPath.2)"
192.168.1.1 - - [18/Nov/2010:15:24:53 +0530] "GET /appl/pix/i/course.gif HTTP/1.1" 304 187 "http://somedomain/appl/course/view.php?id=2" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; InfoPath.2)"
192.168.1.1 - - [18/Nov/2010:15:24:53 +0530] "GET /appl/pix/t/groupn.gif HTTP/1.1" 304 187 "http://somedomain/appl/course/view.php?id=2" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; InfoPath.2)"

I want to have some thing can take relevant lines from log on the basis of date:time so how do I do a grep on that does any one have any simple suggestion in this situation,the log files are pretty big and I need to get only those responses based on time stamps (which can be ranges from users) which gave errors.

share|improve this question
3  
Accept some questions and vote a little.. not many people will keep answering your questions :/ –  Arenstar Nov 18 '10 at 14:28

3 Answers 3

up vote 1 down vote accepted

I'd recommend learning to punctuate your sentences in addition to learning how to use software tools.

I need to get only those responses based on time stamps (which can be ranges from users) which gave errors

But the log sample you've provided does not contain the HTTP status code - is this from the error log? If not then you need to change your log format to something more informative.

'grep' varies between implementations (you don't say what OS you are running this on) but it almost universally does not provide for implementation of a finite state machine - which you'll need if you can't express the time range as a single regular expression (which is how Arenstar's solution works).

If it were me, I'd use awk - something like:

cat logfiles | awk 'BEGIN { output=0; }
/\[18\/Nov\/2010:15/ {
    output=1;
}
/\[19\/Nov\/2010:02/ {
    output=0;
    exit 0; # ... if no further blocks to be extracted
}
{ if (output) print $0; }'

This would return all the log entries between 3pm on the 18th and 2am on the 19th.

behind chain of reverse proxy servers

erm, really?

share|improve this answer
    
I liked this script and the other replies you people gave were informative.I do not know python that well but if this can work then this is great. –  Bond Nov 20 '10 at 7:31
cat apachelog | grep '[18/Nov/2010:15:24'

Should do it.. just change the date/time as you please

share|improve this answer

See the Python script I wrote. I prints log entries based on a range of times. It would need to be adapted to work with Apache logs since it was written for syslog log files.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.