I have been testing my website with a SEO checkup tool and one of the warnings I got was this:

Your server appears to allow access from User-agent Libwww-perl. Botnet scripts that automatically look for vulnerabilities in your software are sometimes identified as User-Agent libwww-perl. By blocking access from libwww-perl you can eliminate many simpler attacks. Read more on blocking Libwww-perl access and improving your website's security.

Afterwards I followed a guide on http://www.seositecheckup.com/articles/87 and created a .htaccess file via Notepad and added the following lines:

RewriteCond %{HTTP_USER_AGENT} libwww [NC,OR]

RewriteCond %{QUERY_STRING} ^(.*)=http [NC]

RewriteRule ^(.*)$ – [F,L]

I then placed the .htaccess file in my website-root (beside my default.asp) and re-tested with the SEO tool. But the same warning comes up once again. Is there anyway for testing this myself or have I done anything wrong?


You did not mention your web server name: apache, iis, or...

The .htaccess files can be used with apache and you have to include a directive like:

AllowOverride FileInfo

Without such a line, the .htaccess file will be ignored.

For the testing issue, you can use a firefox plugin called "User Agent Switcher". This enables you to type your own custom user agent.

| improve this answer | |

You should use this code after "RewriteEngine On" roles in .htaccess :

RewriteCond %{HTTP_USER_AGENT} libwww-perl.*
RewriteRule .* – [F,L]

I use it and it Works! Have a good time

| improve this answer | |
  • 1
    The server required web.configs and not .htaccess - Look at the other answers + comments – Behrens Apr 18 '12 at 9:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.