Good way of dealing with security scanners on a web server

Question

I'm wondering, what would you guys recommend when it comes to deal with people scanning your web server for potential vulnerabilities. Any good approaches on how to stop this?

Would it help to block their IP at the firewall level? I was thinking that perhaps adding an iptables' rule to block or rate the limit of incoming connections might help? If so, what would be a good iptables' rule for this?

Any thoughts about it?

Thanks

Answer 1

Any good approaches on how to stop this?

Ensure that your server is secure, and they'll leave you alone.

Answer 2

There are two stages to dealing with automated bots:

1. Detecting and identifying the bots.
2. Deciding what to do about them.

If you have any kind of moderately popular website, the numbers of bots will make it impractical to do this manually. You will need an automated response to the automated bots.

There's a good chance that a vulnerability scanning bot will produce a lot of 404s (I'm presuming you're talking about web application vulnerabilities and not SSH or Apache vulnerabilities. If you are more concerned about port scanners looking for MySQL exposed to the internet, then a firewall and ignoring them is the best bet.) so you might want to start by creating a script that scans your Apache error log for 404 errors and groups the results by IP address. Something like this:

grep "File does not exist:" /var/log/httpd/error_log | cut -d' ' -f1 | sort | uniq -c | sort -n | tail

Will give you the top 10 highest 404-requesting IP addresses. However, this isn't evidence of any wrongdoing. You also need to look at some of the actual requests from the access log to verify that these really are attempts to probe for vulnerabilities. This script will give you a list of "likely suspects" from which you can start narrowing down the real culprits. Run it from a cron job and use the output in the next step.

The next thing I would do is to start assigning scores, a bit like the way SpamAssassin scores spam emails, to IP addresses based on the requests that it has made. For instance, certain strings crop up a lot in automated attacks.

• URLs that end in ".txt???????"
• URLs that include "?var1=http://evil.com/foo.txt"
• URLs that include "../../../"
• URLs that contain any Javascript, HTML, PHP or SQL.

Any of these (and I'm sure you will find your own strings) should increase the score of an IP address. Of course, attackers can adapt to a certain extent to evade your detection, but the chances are good that they will not do this to you until you become a much higher value target. This strategy will work for a long time to come.

Once you have scored each IP address, find a score threshold that all the bots are higher than and all your real users are lower than and add all the IP addresses that are higher to a list to do something about.

Now we get to the second stage: deciding what to do about them.

You can:

1. Send them an HTTP 403 (Forbidden) message.
2. Send them an ICMP reject packet (This is -j REJECT in your iptables config)
3. Drop their packets and don't send them anything at all. (This is -j DROP in your iptables config)
4. Rate limit their connection to 60 packets per minute or 512 bytes per second.
5. Log everything and contact their ISP.
6. None of the above... there are plenty more options.

All of the above options involve modifying config files and restarting services. I would recommend auto-generating an entire file and using an Include statement in your Apache config or appending it to your iptables config and then restarting Apache or iptables. Be very careful with iptables, it's easy to lock yourself or your users out of your box. Make sure you build a failsafe into the script or you have another method of accessing it such as a console port or a KVM.

Answer 3

Best approach is to stop it before it reaches the web server. While that could be done using a firewal on the web server, I believe the two functions are specialised enough that they should be independent. Any decent firewall will provide the protection you're looking for by responding to a port scan by simply dropping all packets coming from that source and this does not require or generate "millions of entries", as suggested elsewhere.

Answer 4

I have used PSAD on a firewall in the past with good results. You my also us iptables rate limit option to slow down DOS attacks on the webserver itself.

The best option is not to get 4chan ticked at you.

Answer 5

An couple easy methods are blocking ICMP (ping) from anything but your local subnet, and blocking requests that are by IP instead of by DNS.

The first is doable in IPTables. The second is easy with mod_security2.5 provided you are using Apache. (Free, Mod_security is an application level firewall and makes a nice complement to IPTables)

When legitimate clients request pages the request normally contains the DNS information. Bots on the other hand often just increment IPs.

Answer 6

No. There is no way. We do not tlk of people, we mostly talk of bots scanning IP addresses at random.

IpTables - have fun. The next worm will ad millions of entries to your tables.

Just make sure your server is safe and live with it.