We have a dedicated server with a hosting company with a couple of dozens of webs in it. It happens that the nameservers (EG: ns1.domain.com, ns2.domain.com) ip's are coincident with some client webs, ...
We have a clustered storage server that contains many project files like Corel, Indesign, Edius, Adobe Premiere, etc.. Also it has lots of image, audio and video files. We are using sharepoint and ...
Maybe this is a stupid question, but... I am working with this company and they said they needed to get "permission" to crawl other people's sites. They have a Google Search Appliance And some ...
If the robots.txt file is missing in the root directory of a website, how are things treated as: the site is not indexed at all the site is indexed without any restrictions It should logically be ...
Google's indexing bot (edit: yes, it's Google, IP resolves) seems to be adding arbitrary query strings to our home page. xx.xxx.xx.xxx - - [30/Jun/2009:10:14:37 -0400] "GET /?key=61680 HTTP/1.1" 200 ...