Bot detection by fail2ban

Question

I want to know if it is possible to use in fail2ban some rule / script that detects the bots, not just by maxretry in a given amount in seconds, but through identifying some patterns for every IP: for example, let's say that an IP accesses a page every from 10 to 15 seconds, but another IP accesses it every 30-45 seconds.

I have problems with users that use pyautogui scripts and I can not detect the IPs behind the bots because everyone has a different pattern.

Also, I use Sucuri, which has 0% protection in this usecase. I can not switch to another firewall service because this one has only 6 IPs (CloudFlare has over 20, for example) and I have only 10 firewall rules , also maximum IPs, to whitelist in my server provider (I protect myself by attacks through IP, not just by DNS).

Is another tool that can do that? Thanks in advance for help and suggestions!

Best kind regards!

Answer 1

fail2ban is good at detecting known bad patterns that happen repeatedly. Multiple ssh authentication failures match a regex, and banned.

fail2ban is bad at detecting unknown patterns, and has no obvious mechanism for triggering based only on timing. Flagging everything as failed and sorting it out in actions (that you would need to write) seems horrible. False positives all over the place, what about people who tend to click every 15 seconds naturally. Bad for performance, sending all requests through fail2ban jails. And users who want to look less like a bot can mask inhumanly precise timing patterns. wget --random-wait is available in a command line tool, for example.

So your search for a tool continues. You will need to do this selection, we don't do recommendations on Server Fault. Possibly a centralized logging system is appropriate, to parse and store events. Think about the queries it might need to answer, like "list messages containing some IP throughout all our infrastructure". Fancy enough logging tools call themselves security information and event management (SIEM). Even fancier ones with automation workflows have started calling themselves security orchestration, automation and response (SOAR). These might be way too much however, and maybe you just grep log files on an ad-hoc basis when the bots seem to be bad.

---

Single digit sized IP allow list seems small. You already found one service (CloudFlare) that by itself exceeded that. Practical allow lists are not going to get smaller, with ever fragmented IPv4 space, and adding services and applications.