Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Is it possible to create a log file (for example not_found.log) with only 404 HTTP Status Code with Apache2?

Just like this: http://www.helicontech.com/articles/mod_log_config-capabilities/ (Per status code differentiation)

The main problem with this solution is that when status doesn’t match, each log will contain a non-informative record like this (--,- -- -):

cat /var/log/apache2/not_found.log

[28/Aug/2013:16:45:42 +0200] 192.168.1.0.,Opera/9.80 (X11; Linux x86_64) Presto/2.12.388 Version/12.15 /index.php/LogApache/titi 404
[28/Aug/2013:16:45:43 +0200] 88.191.153.141,Opera/9.80 (X11; Linux x86_64) Presto/2.12.388 Version/12.15 /favicon.ico 404 - -,- -- - - -,- -- -

My goal is to count 404 hits and it takes too many time to parse an access_log with more than 1 000 000 lines, that's why I just want to run a wc -l on not_found.log.

Thank you in advance for your help,

Best regards,

share|improve this question

1 Answer 1

up vote 0 down vote accepted

My goal is to count 404 hits and it takes too many time to parse an access_log with more than 1 000 000 lines, that's why I just want to run a wc -l on not_found.log.

graeme@server:~$ wc -l access_log 
10219670 access_log
graeme@server:~$ time awk '$9 == 404' access_log |wc -l
11060

real    0m23.791s
user    0m14.440s
sys     0m2.450s

That's 23 seconds to count 404s in a file with 10,000,000 lines. Not fast enough?

share|improve this answer
    
To be more precise, the file can reach 90 000 000 lines, time awk '$9 == 404' access_log |wc -l 90.40s user 14.28s system 83%. It is also based on your configuration. My main problem is that I have to run my script in less than 1 minute. –  Virtuose Aug 29 '13 at 8:15
    
You would do well to include that sort of information upfront. Saying "more than 1 000 000" when you mean "up to 90 000 000" is far from useful. You also don't mention the time constraint at all in your question. –  ThatGraemeGuy Aug 29 '13 at 9:45

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.