How to create .htaccess and block UserAgent libwww-perl?

Question

I have been testing my website with a SEO checkup tool and one of the warnings I got was this:

Your server appears to allow access from User-agent Libwww-perl. Botnet scripts that automatically look for vulnerabilities in your software are sometimes identified as User-Agent libwww-perl. By blocking access from libwww-perl you can eliminate many simpler attacks. Read more on blocking Libwww-perl access and improving your website's security.

Afterwards I followed a guide on http://www.seositecheckup.com/articles/87 and created a .htaccess file via Notepad and added the following lines:

RewriteCond %{HTTP_USER_AGENT} libwww [NC,OR]

RewriteCond %{QUERY_STRING} ^(.*)=http [NC]

RewriteRule ^(.*)$ – [F,L]

I then placed the .htaccess file in my website-root (beside my default.asp) and re-tested with the SEO tool. But the same warning comes up once again. Is there anyway for testing this myself or have I done anything wrong?

Answer 1

You did not mention your web server name: apache, iis, or...

The .htaccess files can be used with apache and you have to include a directive like:

AllowOverride FileInfo

Without such a line, the .htaccess file will be ignored.

For the testing issue, you can use a firefox plugin called "User Agent Switcher". This enables you to type your own custom user agent.

Answer 2

You should use this code after "RewriteEngine On" roles in .htaccess :

RewriteCond %{HTTP_USER_AGENT} libwww-perl.*
RewriteRule .* – [F,L]

I use it and it Works! Have a good time