up vote 0 down vote favorite
share [g+] share [fb]

I have a standard apache error log file. I would like to see what URLs are causing 404s, since I have moved this site around and I want to find bad links. Can anyone recommend a bash snippet that will parse this log using awk or something to show me the popular 404s?

I know there are advanced programmes for this sort of thing. I'm just looking for something simple.

link|improve this question
79% accept rate
feedback

2 Answers

up vote 6 down vote accepted

This should do it:

grep ' 404 ' /var/log/apache2/access.log | cut -d ' ' -f 7 |sort |uniq -c |sort -n
link|improve this answer
Add a "| sort -n" to get them in order of hits – David Pashley Jun 30 '09 at 10:18
I agree with this solution with one important side note: This works with most Apache installations that use the system default logging format. If the admin has changed the logging format the command above must be altered slightly. – Niels Basjes Jun 30 '09 at 12:35
feedback
up vote 3 down vote

An awk answer :

awk '$9 == 404{urls[$7]++}END{for (url in urls) print urls[url] "\t" url}' access_log | sort -n

It's just for fun as it's probably much much slower than womble solution

link|improve this answer
Got in just as I was writing mine. – LapTop006 Jun 30 '09 at 11:40
Have you a nice solution to include the sort -n in awk ? I did not found a way to do it in a simple way – radius Jun 30 '09 at 12:01
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.