Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Yandex.ru spiders are known of disobeying robots.txt. In Apache, you can disable yandex in htaccess file. How can I do the same thing with Tomcat? I'm using ubuntu server, so I thoght about appropriate entries in hosts.deny, but I'm not sure if

ALL: yandex.ru

will be sufficient and efficient, cause bot's are often named like spider31.yandex.ru. Could you show example of blocking this bots via hosts.deny (or any other way)?

share|improve this question
add comment

1 Answer

You want to put this in hosts.deny:

ALL: .yandex.ru

Notice the . which means any subdomain of.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.