How can I automatically ban an IP requesting the same file too often?

Question

On my site, I serve a zip file which is about 65MB. I currently have an Apache rule setup to allow only 2 simultaneous downloads, showing a 503 error if you try to download more. Lately, after looking through the access logs, some IPs are getting the 503 error at about 3 requests per second. Once the download finishes, however, they can download it again. After further investigation, the IP seems to be simply downloading for the heck of it. This is a waste of bandwidth and I was wondering how I would temporarily ban an IP that causes a certain error code too often within a given time range.

I'd like to be able to ban it at the iptables level. The server is running Ubuntu LTS (8.04)

Answer 1

I'm not an expert at this kind of things, but I recall having heard of fail2ban once or twice. It might do something like what you want.

(But looks not that easy to configure... You'll defintly have to test on some "testing" server before depolying it to your production box ^^ )

Answer 2

Maybe side-stepping the problem, but perhaps you could host the file with a free or cheap(er) service; hopefully they will have mechanisms which prevent abuse like this. I only suggest in order to buy you time before you settle on a better solution. :-)

Answer 3

Play with something like this:

iptables -I INPUT -p tcp --dport 80 -m state --state NEW -m recent --update --seconds 60 --hitcount 4 -j DROP

Answer 4

Adding it to hosts.deny get you what you want. You can automate this kind of thing with software like blockhosts and denyhosts.