1
vote
6answers
2k views

What happens if a website does not have a robots.txt file?

If the robots.txt file is missing in the root directory of a website, how are things treated as: the site is not indexed at all the site is indexed without any restrictions It should logically be ...
1
vote
3answers
539 views

odd query strings in Googlebot requests

Google's indexing bot (edit: yes, it's Google, IP resolves) seems to be adding arbitrary query strings to our home page. xx.xxx.xx.xxx - - [30/Jun/2009:10:14:37 -0400] "GET /?key=61680 HTTP/1.1" 200 ...