Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have been testing my website with a SEO checkup tool and one of the warnings I got was this:

Your server appears to allow access from User-agent Libwww-perl. Botnet scripts that automatically look for vulnerabilities in your software are sometimes identified as User-Agent libwww-perl. By blocking access from libwww-perl you can eliminate many simpler attacks. Read more on blocking Libwww-perl access and improving your website's security.

Afterwards I followed a guide on and created a .htaccess file via Notepad and added the following lines:

RewriteCond %{HTTP_USER_AGENT} libwww [NC,OR]

RewriteCond %{QUERY_STRING} ^(.*)=http [NC]

RewriteRule ^(.*)$ – [F,L]

I then placed the .htaccess file in my website-root (beside my default.asp) and re-tested with the SEO tool. But the same warning comes up once again. Is there anyway for testing this myself or have I done anything wrong?

share|improve this question
up vote 4 down vote accepted

You did not mention your web server name: apache, iis, or...

The .htaccess files can be used with apache and you have to include a directive like:

AllowOverride FileInfo

Without such a line, the .htaccess file will be ignored.

For the testing issue, you can use a firefox plugin called "User Agent Switcher". This enables you to type your own custom user agent.

share|improve this answer
Ah, it is IIS7. Which might be the reason for why it doesnt work :/ – Behrens Jan 4 '12 at 16:02
Have a look at this link:… – Khaled Jan 4 '12 at 16:09
Thanks mate! :) – Behrens Jan 5 '12 at 12:33

You should use this code after "RewriteEngine On" roles in .htaccess :

RewriteCond %{HTTP_USER_AGENT} libwww-perl.*
RewriteRule .* – [F,L]

I use it and it Works! Have a good time

share|improve this answer
The server required web.configs and not .htaccess - Look at the other answers + comments – Behrens Apr 18 '12 at 9:08

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.