Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm in the processing of setting up a Google Mini device to index our site which has a lot of dynamically generated content. I've created a dynamic file which lists all of the dynamic URL's. This is currently being indexed by Google but seems to be ignored by the Google mini device.

I've added the file into the "Start crawling from the following URL's". When I view the crawl diagnostics the file comes up but none of the dynamic URL's contained within the are being indexed. The Google mini device is only indexing 100 URL's whereas the contains 10,000.

If I search for a phrase using the test centre the search results includes the and not the URL it points to.

Any ideas?

share|improve this question
Are the Google Mini's really blue? – Mark Henderson Sep 3 '09 at 2:58
Yes, both the box and the feeling it's giving me! – Dave Barker Sep 3 '09 at 3:18
Cool. Well, not the headache it's giving you, but a blue server, would stand out. – Mark Henderson Sep 3 '09 at 4:19
up vote 1 down vote accepted

We've just had a consultant come in who has stated the Google Mini cannot index the URL's contained in a sitemap.xml file. One alternative solution is to create a HTML page with all of the links within it.

share|improve this answer

A couple of things from the top of my head:

  1. Make sure the content type of your site map file is allowed to be indexed. html for example is allowed by default.

  2. Make sure your site map is in the XML site map format.

  3. If you have multiple collections defined, make sure that your site map file is reachable by the collection you expect it to be.

share|improve this answer
The site map is called and is being indexed so I think that rules out #1. The file is being indexed not the URL's it contains. It conforms to #2 and has been submitted and indexed by the main Google search engine. We've only got 1 collection so that rules out #3. Any other suggestions? – Dave Barker Sep 3 '09 at 4:00

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.