I have installed Fail2Ban v0.10.2 on Ubuntu 18.04 with Apache 2.4.29 and enabled the standard ssh and apache jails for basic protection with email notification warnings, when an IP is blocked.

Having a look at the documentation, I was not able to find a relevant filter that would help with the following situation:

I would like to ban IPs that hit the server and produce large numbers of 404 errors due to fake URL requests, which can be a typical spam bot behavior. So ideally, an IP is blocked that produces more than three 404 errors in a row with some exceptions for official search engine crawlers.

Is there a default regex for this situation?

I would appreciate your assistance on how to implement this.


I recommend you start by implementing the built-in apache-noscript filter for fail2ban. To do so, add the following lines to/etc/jail.local`

     enabled = true 
     port = http,https 
     filter = apache-noscript 
     logpath = /var/log/apache2/*error.log 
     maxretry = 3 
     bantime = 600 

tweak the bantime setting to your liking and consider implementing the recidiv filter/jail for repeat offenders.

Note: there is a possible bug with the filter regex

| improve this answer | |

I recommend configuring apache to not log anything around these locations that you frequently get 404 message on. That way the CPU and disk IO saved writing the logs can be used for your real visitors.

CPU/IO time is also saved when you don't need fail2ban to scan through the logs.

Every real visitor is saved from being subject to IP/nftables rules slowing down their access.

You'll also be saved the anguish of looking at the logs and focusing on the background noise of the internet rather than the real visitors you care about.

| improve this answer | |
  • 1
    PS, don't take me for a fail2ban hater, I was a core maintainer for a number of years before the number of requests like this where too much and distracting from writing real security benefits so I quit. – danblack Sep 15 '18 at 11:17
  • 3
    I run a number of websites and do not 'frequently' get 404s. The majority of 404s are people attempting to find exploits on the websites and believe it is perfectly reasonable to block multiple attempts that result in a 404 – ChrisBint Dec 1 '18 at 15:17
  • 3
    That's actually very bad advice. If someone is trying to break your website, there should be traces and warnings about it, not silent ignores, because "real visitors". If your server has trouble logging errors, then you should buy a better server, not ignore this problem. – quamis Mar 20 '19 at 7:19
  • 404s are harmless. Automaticly classing them as potential threat is a stretch considering most are just automated and even the concept they could do damage is remote. Deploying f2b for low end noise distracts from serious effort in hardening and vulnerability in the same way AV acts as a deficient placebo for managing malware risk. This is a really hard concept to grasp when you're invested in classifying 404 messages as threats and gain euphoria from seeing bans and less log messages. This relief is a distraction from doing stronger access controls and code review that will actually protect. – danblack Mar 20 '19 at 7:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.