From time to time, I see collections of suspect "File not found" errors in my Apache logs, basically using the pattern

File does not exist: /var/www/file, referer: http://my.server.com/file

In human terms: The file was not found, though it referenced here itself. A clear hacking attempt, as that's hardly possible (and the REQUEST_URIs often enough suggest the same). In my eyes a clear case for fail2ban – if I could get backreferences to work here:

failregex = ^%(_apache_error_client)s File does not exist: /var/www(.+), referer: http://.+\1$

(Justin Case: above examples assume the DIRECTORY_ROOT of that webserver being /var/www)

I googled for hours, searched the fail2ban wiki up and down – but nowhere I could find a statement concerning backreferences in its filters. Are they not supported, or did I do it the wrong way? Any hints how to make it work (except from "dirty hacks" like first sending the request to another fake url using , and then catching on that (if anyone is interested, I can elaborate on that approach in an answer), or doing something similar using )?

as an entire log line was requested:

[Fri Nov 08 14:57:28 2013] [error] [client] File does not exist: /var/www/text/files.htm++++++++++++++++++++++++++Result:+using+proxy+;+no+post+sending+forms+are+found;, referer: http://www.myserver.com/text/files.htm++++++++++++++++++++++++++Result:+using+proxy+;+no+post+sending+forms+are+found;

(sorry, logs were just switched, so this long candidate was the only one left currently; minor adjustments were made for privacy reasons)

  • Would you post an entire line from the log? – rickhg12hs Nov 9 '13 at 2:05
  • Sure. Just added one to the end of my question. Though it adds no new information: except for the different file path, it's just prefixed with the default Apache Error log information which looks the same with each Apache install ;) – Izzy Nov 9 '13 at 12:44
  • fail2ban uses normal python regexes, so backreferences should work. Do you get a regex compilation error or do things simply not match? What does fail2ban-regex say? – Dennis Kaarsemaker Nov 9 '13 at 12:50
  • That's exactly the thing. I've checked the regex, and it works outside fail2ban. fail2ban-regex simply finds no hits. I tried to narrow it down: fail2ban-regex shows matches if I leave the trailing \1$ off the line; but as soon as I add those 3 chars (or just the \1), no more hits. – Izzy Nov 9 '13 at 12:52
  • Just for the records: seems not to be a "hacker", but rather a SEO tool behind those requests (see e.g. here). Still, those are not welcome visitors for some of us ;) – Izzy Nov 10 '13 at 13:43

Looks like the backreference numbering is a bit wacky. Try using a Named Group ... something like this:

failregex = ^%(_apache_error_client)s File does not exist: /var/www(?P<snoop_file>.+), referer: http://.+(?P=snoop_file)$
| improve this answer | |
  • 2
    Excellent! That did the trick. Be gone, ya SEO crawlers :) And big thanks, rick (+1 & accepted)! – Izzy Nov 11 '13 at 0:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.