Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm trying to parse apache log files, but I'm finding some strange results and I'm not sure what they mean. Hopefully someone can provide some insight. (all of the IP addresses were altered. none actually start with 192, I didn't figure the search engines mattered though.)

In the first example, multiple ip addresses are showing up in the host field:

192.249.71.25 - - [04/Aug/2009:04:21:44 -0500] "GET /publications/example.pdf HTTP/1.1" 200 2738
192.0.100.93, 192.20.31.86 - - [04/Aug/2009:04:21:22 -0500] "GET /docs/another.pdf HTTP/1.0" 206 371469

What causes this? Does it have to do with proxy servers? Is there a way to have Apache only log one?

Edit:

Here it is:

LogFormat "%h %l %u %t \"%r\" %s %b \"%{Referer}i\" \"%{User-agent}i\" %I %O" common

In the second example, a bunch of information is just completely missing! What would cause this?

msnbot-65-55-207-50.search.msn.com - - [29/Dec/2009:15:45:16 -0600] "GET /publications/example.pdf HTTP/1.1" 200 3470073 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)" 266 3476792
- - - - "-" - - "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; InfoPath.1)" 285 594
- - - - "-" - - "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; InfoPath.1)" 285 4195
- - - - "-" - - "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; InfoPath.1)" 299 109218
crawl-17c.cuil.com - - [29/Dec/2009:15:45:46 -0600] "GET /publications/another.pdf HTTP/1.0" 200 101481 "-" "Mozilla/5.0 (Twiceler-0.9 http://www.cuil.com/twiceler/robot.html)" 253 101704

My CustomLog configuration says:

LogFormat "%h %l %u %t \"%r\" %s %b \"%{Referer}i\" \"%{User-agent}i\" %I %O" common
share|improve this question

migrated from stackoverflow.com Jan 13 '10 at 21:53

This question came from our site for professional and enthusiast programmers.

    
We need your CustomLog directive to determine what you've done. –  womble Jan 13 '10 at 22:41
    
+1 to womble's comment ... you've posted a LogFormat definition but we need your CustomLog line in order to determine what logging format has been applied to your config. –  DaveG Feb 27 '10 at 9:45

3 Answers 3

up vote 1 down vote accepted

It looks to me like you are behind a proxy/cache and you are using mod_rpaf which replaces the IP address that apache sees in %h with the contents of X-Forwarded-For. Some proxy servers include their IP and the forwarded IP (giving you the 1.2.3.4,2.3.4.5 IP addresses. The reason you are getting some blank results is that some proxy servers use headers other than X-Forwarded-For and rpaf replaces a null value with -.

share|improve this answer

Can you post the output of httpd -l and httpd -M, please?

That comma separated %h line looks suspiciously like something is jamming in an X_FORWARDED_FOR value. The fact that you don't have a date on the subsequent lines seems odd, so my assumption is that there is a module loaded that's registered a logging hook and it's munching up your output.

I've never seen that happen and I run some pretty large Apache installations. It has got to be some screwy module causing problems (or, perhaps the lack of a required module).

The other possibility might be that Apache doesn't know how to satisfy your format directive. The '-' simply means that a requested data point wasn't available. Those format specifiers are built in to mod_log_config, which ought (needs?) to be present by default.

Edit... I know Ubuntu & Red Hat define a 'combined' as well as a 'common.' Could you try using that format instead? I think someone already mentioned that, though.

share|improve this answer

It looks like you are reusing the common logging definition. This probably isn't supported. Try defining your own logger

LogFormat "%h %l %u %t \"%r\" %s %b \"%{Referer}i\" \"%{User-agent}i\" %I %O" mylogformat
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.