0

Is it possible to create a log file (for example not_found.log) with only 404 HTTP Status Code with Apache2?

Just like this: http://www.helicontech.com/articles/mod_log_config-capabilities/ (Per status code differentiation)

The main problem with this solution is that when status doesn’t match, each log will contain a non-informative record like this (--,- -- -):

cat /var/log/apache2/not_found.log

[28/Aug/2013:16:45:42 +0200] 192.168.1.0.,Opera/9.80 (X11; Linux x86_64) Presto/2.12.388 Version/12.15 /index.php/LogApache/titi 404
[28/Aug/2013:16:45:43 +0200] 88.191.153.141,Opera/9.80 (X11; Linux x86_64) Presto/2.12.388 Version/12.15 /favicon.ico 404 - -,- -- - - -,- -- -

My goal is to count 404 hits and it takes too many time to parse an access_log with more than 1 000 000 lines, that's why I just want to run a wc -l on not_found.log.

Thank you in advance for your help,

Best regards,

Improve this question

1 Answer 1

Reset to default
0

My goal is to count 404 hits and it takes too many time to parse an access_log with more than 1 000 000 lines, that's why I just want to run a wc -l on not_found.log.

graeme@server:~$ wc -l access_log 
10219670 access_log
graeme@server:~$ time awk '$9 == 404' access_log |wc -l
11060

real    0m23.791s
user    0m14.440s
sys     0m2.450s

That's 23 seconds to count 404s in a file with 10,000,000 lines. Not fast enough?

Improve this answer
2
  • To be more precise, the file can reach 90 000 000 lines, time awk '$9 == 404' access_log |wc -l 90.40s user 14.28s system 83%. It is also based on your configuration. My main problem is that I have to run my script in less than 1 minute.
    – Virtuose
    Aug 29, 2013 at 8:15
  • You would do well to include that sort of information upfront. Saying "more than 1 000 000" when you mean "up to 90 000 000" is far from useful. You also don't mention the time constraint at all in your question.
    – ThatGraemeGuy
    Aug 29, 2013 at 9:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .