0

"Pre-Fetching" might not be the most widely-used term, but in any event, i'm referring to browser plugins that request pages prior to an actual request from the user. (For instance, when a new page loads and you begin reading it, the Browser will pre-fetch all of the links on that page (since it doesn't know which one you will decide to click).

The Firefox plug-in, Fasterfox, and Google's Web Accelerator (no longer available to download, it seems) are the two such plug-ins that i'm aware of.

Are such requests distinguishable in the Apache Event Log from ordinary page requests?

Improve this question

2 Answers 2

Reset to default
1

Firefox prefetch requests have the HTTP header X-moz set as follows

'X-moz: prefetch'

You could therefore identify prefetches from a CGI script or similar, but not from an apache log file in the default log format. If you define a custom log file format, you could include the contents of the X-moz header, where present:

%...{X-moz}i

So, adjusting the default log format to include this at the end of log lines:

LogFormat "%h %l %u %t \"%r\" %>s %b %{X-moz}i"

There is some work on standardising prefetching in HTML5, but there is currently nothing in the HTML5 draft which requires or suggests the UA identifies such requests to the server.

See the mozilla developer's FAQ on prefetching for more information:

https://developer.mozilla.org/en/Link_prefetching_FAQ

Improve this answer
3
  • thanks jmtd. I've got an Apache Webserver on my MacBook Pro, so i just modified the event log format i/a/w your answer above--works just fine (20 min surfing on FF, approx. 70% of the lines in the logfile are prefetches). [Note: since our rep doesn't xfer from SO, i can't upvote this yet, but will do so shortly.]
    – doug
    Jan 11, 2010 at 16:04
  • Thanks doug, I'd appreciate it - When I signed up here I did carry rep from SO across (which let me upvote and comment etc.) but it disappeared shortly afterwards. I think perhaps they changed their minds.
    – jmtd
    Jan 18, 2010 at 14:12
  • yeah, that's what i heard from someone--that the SO policy had changed. Anyway, you just got your belated upvote from me.
    – doug
    Jan 20, 2010 at 22:55
0

Since they'd probably use the same user-agent as regular page requests, it would be hard to do. You might be able to write something to distinguish it by the timing patterns -- a request for one page followed by a blast of requests for other pages, faster than a human would be likely to request. However, you'd probably get a lot of false positives in there.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .