Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have been testing my website with a SEO checkup tool and one of the warnings I got was this:

Your server appears to allow access from User-agent Libwww-perl. Botnet scripts that automatically look for vulnerabilities in your software are sometimes identified as User-Agent libwww-perl. By blocking access from libwww-perl you can eliminate many simpler attacks. Read more on blocking Libwww-perl access and improving your website's security.

Afterwards I followed a guide on http://www.seositecheckup.com/articles/87 and created a .htaccess file via Notepad and added the following lines:

RewriteCond %{HTTP_USER_AGENT} libwww [NC,OR]

RewriteCond %{QUERY_STRING} ^(.*)=http [NC]

RewriteRule ^(.*)$ – [F,L]

I then placed the .htaccess file in my website-root (beside my default.asp) and re-tested with the SEO tool. But the same warning comes up once again. Is there anyway for testing this myself or have I done anything wrong?

share|improve this question
add comment

2 Answers

up vote 4 down vote accepted

You did not mention your web server name: apache, iis, or...

The .htaccess files can be used with apache and you have to include a directive like:

AllowOverride FileInfo

Without such a line, the .htaccess file will be ignored.

For the testing issue, you can use a firefox plugin called "User Agent Switcher". This enables you to type your own custom user agent.

share|improve this answer
    
Ah, it is IIS7. Which might be the reason for why it doesnt work :/ –  Behrens Jan 4 '12 at 16:02
5  
Have a look at this link: learn.iis.net/page.aspx/557/… –  Khaled Jan 4 '12 at 16:09
    
Thanks mate! :) –  Behrens Jan 5 '12 at 12:33
add comment

You should use this code after "RewriteEngine On" roles in .htaccess :

RewriteCond %{HTTP_USER_AGENT} libwww-perl.*
RewriteRule .* – [F,L]

I use it and it Works! Have a good time

share|improve this answer
1  
The server required web.configs and not .htaccess - Look at the other answers + comments –  Behrens Apr 18 '12 at 9:08
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.