Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I work for a site that often gets attacked by bot networks. We have started to use this tool: which auto-bans IPs that have more open connections than the set value. By default it's set to 150, we are currently using 250.

I would like to know, how low is safe so that search bots and normal visitors do not get blocked?

share|improve this question
up vote 4 down vote accepted

I would like to know, how low is safe so that search bots and normal visitors do not get blocked?

As low as you can get without search bots and normal visitors being blocked. (i.e. "There's no way to tell without empirical data - It depends on your site, the kind of browsers/crawlers being used and how many simultaneous connections they'll attempt to open, whether or not users are behind proxies/NATs that make many users appear to come from one IP address, etc. etc. etc.")

Practical advice: If you want to use an auto-ban like this err on the side of false negatives (allowing attacks to continue) rather than false-positives (banning legit users). 200 simultaneous connections from one IP seems to be a reasonable value, provided you don't have hundreds of users behind a proxy somewhere all looking like one IP and all hitting the site at once.

share|improve this answer
Thanks for the answer. Someone else just suggested putting these IPs: on the whitelist. – Mr.Boon Jan 5 '12 at 23:39
Whitelisting those IPs will keep you from blocking the Google crawler, but it won't do anything to help your users, yahoo's crawler, etc. – voretaq7 Jan 6 '12 at 23:57

A typical browser will not make more than a dozen connections at a time. The problem you'll have however is people behind NATs, particularly large networks, where a dozen people connecting at the same time might send the concurrent connection max over a hundred.

There's really no great answer for this Question. The best we can say is to try and see. You may be able to setup two levels, at 250 they're blocked and at some new proposed limit (say 100) they're logged so you can review the log and see if any legitimate traffic every hits that number.

If there's any other log showing some sort of erroneous activity by the bots you may benefit from using Fail2Ban (or something similar). Say if they're constantly requesting web pages that don't exist you could monitor the logs for 404 statuses.

share|improve this answer

To Quote @chopper3


However in all seriousness there isn't a definitive answer. It depends on several factors such as type of site and content and people behind NAT/large networks. Your best way to choose is to test a proposed setting and see if any legitimate traffic is blocked.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.