Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Can someone help me with a shell script to figure out the number of 500 HTTP response entries in an access log within a time frame specified?

share|improve this question

2 Answers 2

You can use awk to filter in specified time range:

# awk '$9 == "500" && $4 <= to && $4 >= from { print $0 }' from="[02/Aug/2011:14:30:00 +0700]" to="[02/Aug/2011:14:32:00 +0700]" /path/to/your/access_log | wc -l

The status code and timestamp fields may have different order. Also change from and to to corresponding format which you are using.

share|improve this answer
1  
Thanks @quanta. This seems to work if the dates are in the same month, but not when the dates go beyond 1 month. for e.g. ls acc* | while read line; do awk '$9 == "500" && $4 <= to && $4 >= from { print $0 }' from="[31/July/2011:00:00:00]" to="[02/August/2011:00:00:00]" "${line}" | wc -l ; done is 0 for me. –  Quintin Par Aug 3 '11 at 6:01

OK. You can convert to Epoch time to compare:

#!/bin/bash

from=$(date -d "$(echo "$1" | awk 'BEGIN { FS = "[/:]"; } { print $1" "$2" "$3" "$4":"$5":"$6 }')" +%s)
to=$(date -d "$(echo "$2" | awk 'BEGIN { FS = "[/:]"; } { print $1" "$2" "$3" "$4":"$5":"$6 }')" +%s)

while read line
do
    date=$(echo $line | awk '{ print substr($4, 2, length($4)-1) }' | awk 'BEGIN { FS = "[/:]"; } { print $1" "$2" "$3" "$4":"$5":"$6 }')
    date=$(date -d "$date" +%s)
    [[ $date -ge $from && $date -le $to ]] && echo $line
done < $3

and call it with something like this:

./log_filtering.sh 30/Jul/2011:15:55:44 02/Aug/2011:01:00:00 access_log

I'm trying to write in one-line.


Doing with awk:

#!/bin/awk -f

function toEpoch(t, a) {
    split(t, a, "[/:]")
    split("Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec", monthname, " ")
    for (i=1; i<=12; i++) month[monthname[i]] = i
    a[2] = month[a[2]]
    return(mktime(a[3]" "a[2]" "a[1]" "a[4]" "a[5]" "a[6]))
}

BEGIN {
    start = toEpoch(starttime)
    end = toEpoch(endtime)
}

{ date = toEpoch(substr($4, 2, length($4)-1)) }
( date >= start ) && ( date <= end )

and passing the arguments with -v:

gawk -f log_filtering.awk -v starttime=30/Jul/2011:04:12:24 -v endtime=02/Aug/2011:04:12:27 access_log
share|improve this answer
    
I've edited my post to append an awk script. –  quanta Aug 4 '11 at 9:19

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.