According to a recent report from ZDNet,
31% of traffic is illegitimate (hackers, scrapers, spammers). The challenge as a sysadmin is to find a way to block that traffic without blocking your real users and search engines. I've searched
"block traffic" and only found answers about specific tools (like this and this one for
iptables) which doesn't give you an overview of the solutions that are out there and when to use them hence why I would like to post this question in an attempt to gather an authoritative answer on the topic.
I started diving into it and my general understanding is as follows:
-the lower in the stack you check (e.g.
iptables) the better for performance (because it doesn't get processed by the application) but the more likely to block legitimate traffic (e.g. if you filter on IP and a hacker is using an access point you will end up blocking all the users that rely on the same access point).
-the higher in the stack you check (e.g.
apache mod_security) the worse for performance (because every component up to that point has had to process the information) but the less likely to block legitimate traffic.
Since we're running
REHL 5.6 and
Ubuntu 10.04 with
httpd/apache + PHP I'm not going to lie and confess that I am only interested in solutions that fit these configurations.
Q: Could you share, from experience, the tools that you're using (and why) to block illegitimate traffic to your websites?