List top 404 URLs in Apache access_log

Question

I'm looking for a command to list the 10 or 20 top 404 error URLs out of my apache access_log. Can anyone help me with this?

thanks

Answer 1

Assuming the normal access_log format this should do it:

cat access_log | awk '{ if($9 == 404) { print $7 } }' | sort | uniq -c | sort -r | head -10

Answer 2

I've done this using standard unix utilities, awk, sort, etc, and it works fairly well. Since the format of your logs could be different, you may have to change some things to work in your environment, but the basic command would be this:

cat access_log | awk '/" 404 / {print $7}' | sort | uniq -c | sort -n | tail -n10

If you are not familiar with awk, what we are doing is:

for each line
   if it contains the string '" 404', then
     print the 7th field (where fields are space delimited) 

If you are using a custom apache log format, then you will need to change the $7 to match the field for the URL in the GET request. You can also change the number of lines from the tail command to display more, or rewer, results.