0

I have this robots.txt file:

User-agent: *
Sitemap: Path_to_sitemap.xml

My Q is, should I have something else in there as well? Like allow All or something?

Thanks

2 Answers 2

2

It is a valid robots.txt. You may read this post from google: http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449

0

Depends on what you want ;-) You don´t have to allow sth. special, just block the things you wouldn´t have crawled by Google and Co.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .