Tagged Questions

0

votes

2answers

82 views

Is this a valid robots.txt file?

I have this robots.txt file: User-agent: * Sitemap: Path_to_sitemap.xml My Q is, should I have something else in there as well? Like allow All or something? Thanks

asked Sep 30 '10 at 7:05

Anonymous12345
4531414

newest robots.txt browser questions feed

Technology			Life / Arts	Culture / Recreation	Science	Other
Stack Overflow Server Fault Super User Web Applications Ask Ubuntu Webmasters Game Development TeX - LaTeX	Programmers Unix & Linux Ask Different (Apple) WordPress Answers Geographic Information Systems Electrical Engineering Android Enthusiasts IT Security	Database Administrators Drupal Answers SharePoint User Experience Mathematica more (14)	Photography Science Fiction & Fantasy Seasoned Advice (cooking) Home Improvement more (13)	English Language & Usage Skeptics Mi Yodeya (Judaism) Travel Arqade (gaming) Bicycles Role-playing Games more (22)	Mathematics Cross Validated (stats) Theoretical Computer Science Physics more (7)	Stack Apps Meta Stack Overflow Area 51 Stack Overflow Careers

rev 2013.5.17.690

Tagged Questions

Is this a valid robots.txt file?

Community Bulletin

Related Tags