Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I'm trying to detect bad bot activity. If the information.php, which is disallowed in robots.txt but is open for normal users, has been requested more than 500 times in a day by same IP address, I'd say its a safe bet to mark the user as a bot.

Also, if I get more than 100 bad login requests from an IP address, I'd like to mark that as bot activity. (although I think I'd be better off presenting the user with a captcha after 1 bad login attempt, but an IP-based solution will be a bit difficult to implement than a session cookie based solution IMO)

And finally, I'd like to block the IPs that have been marked for bot activity. How do I go about doing all this?

share|improve this question
    
You forgot to ask a question. –  David Schwartz Jun 21 '13 at 10:25
    
question is, how? –  Peeyush Kushwaha Jun 21 '13 at 13:28
    
Just a comment: Captcha after 1 failed login attemp? Ugh. Push it to something like 5, because the regular users tend to forget password or have to type in multiple combinations as, for example, sites password policy forced them to use lower, upper and digits. –  Janis Veinbergs Jun 21 '13 at 13:58

4 Answers 4

You should have a look at fail2ban which parses log files (such as apache, ssh, and so on) and takes decision (blocking for a while by iptables) to suspicious users.

share|improve this answer

What you can opt for is to use a Host Based Intrusion Detection system which parses logs and automatically blocks an account for a certain period. Examples are Fail2ban and OSSEC.

Furthermore OSSEC features a server client model so you can have a centralized model which allows for better manageability should you have more than one webserveR.

share|improve this answer

I had to implement the same solution few days ago and fail2ban is right tool for this task. There is good how-to: Fail2ban protect web server http DOS attack

Just modify failregex for your case.

share|improve this answer

It sounds like you want mod_security. mod_security lets you use, or write powerful rules to to detect and react on activity on your web server.

It is immensely powerful, and should suit most of your needs without too much trouble.

It is also widely used, so getting help should be easy.

http://www.modsecurity.org/projects/modsecurity/

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.