I want to disallow /path but also wanna allow /path/another-path in robots.txt. I already tried:

Disallow: /path


Disallow: /path$ 

But doesn't work, I mean it blocked /path/another-path too. Is it possible to do that? Any help would be appreciated.

migrated from stackoverflow.com Oct 8 '12 at 14:05

This question came from our site for professional and enthusiast programmers.


You could use Allow. But note that it's not part of the original "specification", however, some search engines support it.

User-agent: *
Allow: /path/another-path
Disallow: /path/

See http://en.wikipedia.org/wiki/Robots_exclusion_standard#Allow_directive, for example.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy