Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

"Pre-Fetching" might not be the most widely-used term, but in any event, i'm referring to browser plugins that request pages prior to an actual request from the user. (For instance, when a new page loads and you begin reading it, the Browser will pre-fetch all of the links on that page (since it doesn't know which one you will decide to click).

The Firefox plug-in, Fasterfox, and Google's Web Accelerator (no longer available to download, it seems) are the two such plug-ins that i'm aware of.

Are such requests distinguishable in the Apache Event Log from ordinary page requests?

share|improve this question

2 Answers 2

up vote 1 down vote accepted

Firefox prefetch requests have the HTTP header X-moz set as follows

'X-moz: prefetch'

You could therefore identify prefetches from a CGI script or similar, but not from an apache log file in the default log format. If you define a custom log file format, you could include the contents of the X-moz header, where present:

%...{X-moz}i

So, adjusting the default log format to include this at the end of log lines:

LogFormat "%h %l %u %t \"%r\" %>s %b %{X-moz}i"

There is some work on standardising prefetching in HTML5, but there is currently nothing in the HTML5 draft which requires or suggests the UA identifies such requests to the server.

See the mozilla developer's FAQ on prefetching for more information:

https://developer.mozilla.org/en/Link_prefetching_FAQ

share|improve this answer
    
thanks jmtd. I've got an Apache Webserver on my MacBook Pro, so i just modified the event log format i/a/w your answer above--works just fine (20 min surfing on FF, approx. 70% of the lines in the logfile are prefetches). [Note: since our rep doesn't xfer from SO, i can't upvote this yet, but will do so shortly.] –  doug Jan 11 '10 at 16:04
    
Thanks doug, I'd appreciate it - When I signed up here I did carry rep from SO across (which let me upvote and comment etc.) but it disappeared shortly afterwards. I think perhaps they changed their minds. –  jmtd Jan 18 '10 at 14:12
    
yeah, that's what i heard from someone--that the SO policy had changed. Anyway, you just got your belated upvote from me. –  doug Jan 20 '10 at 22:55

Since they'd probably use the same user-agent as regular page requests, it would be hard to do. You might be able to write something to distinguish it by the timing patterns -- a request for one page followed by a blast of requests for other pages, faster than a human would be likely to request. However, you'd probably get a lot of false positives in there.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.