Is there any way to write rules for robots.txt or htaccess that will block all bots that come from a .ru domain? Thanks
1 Answer
Yes you can with htaccess and some regex. A quick bit of googling shows up many examples, you could have done that yourself before asking here...
http://httpd.apache.org/docs/1.3/howto/auth.html#allowdeny
For example:
Order deny,allow
Deny from .ru
Allow from all
However this is reliant on a genuine referrer and therefore can be bypassed. For a more robust method of blocking countries or TLD's you should implement a separate security appliance.
-
So adding something like this: <code>deny from .ru</code> will block all .ru user agents?– bobAug 12, 2015 at 18:42
-
1@bob, you have to specify the Order then specify the Deny and Allow rules. However, this is very likely a bad idea. This is going to require a lot of reverse DNS lookups, and if your system sees a lot of traffic it has the potential for slowing things down greatly.– GeneAug 12, 2015 at 19:45
-
Thanks for that info, I never would have guessed it. I'll leave it as is now.– bobAug 12, 2015 at 19:56
mod_evasive
. You can configure apache to block a client if it attempts too many requests within a given time frame. linode.com/docs/websites/apache-tips-and-tricks/…