Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I want to disallow /path but also wanna allow /path/another-path in robots.txt. I already tried:

Disallow: /path

Or:

Disallow: /path$ 

But doesn't work, I mean it blocked /path/another-path too. Is it possible to do that? Any help would be appreciated.

share|improve this question
add comment

migrated from stackoverflow.com Oct 8 '12 at 14:05

This question came from our site for professional and enthusiast programmers.

1 Answer

You could use Allow. But note that it's not part of the original "specification", however, some search engines support it.

User-agent: *
Allow: /path/another-path
Disallow: /path/

See http://en.wikipedia.org/wiki/Robots_exclusion_standard#Allow_directive, for example.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.