Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

There are 3000 entries inside sitemap.xml,but it turns out that Google just crawls 300 of them,what's the problem?

share|improve this question

migrated from Oct 16 '09 at 11:12

This question came from our site for professional and enthusiast programmers.

because 3000 is a WOW amount. – Mauris Oct 16 '09 at 6:47
I don't think it's in the number of entries. According to this Wikipedia article you can point to 5000 * 1000 entries. – Luke Oct 16 '09 at 6:52

I'd suggest taking a look into Google Webmaster Tools. There you see the number of submitted URLs and the number of indexed URLs. If URLs get indexed depends on links back to the page, so they might not get indexed even if they're crawled (note the difference). Furthermore, Webmaster Tools tells you if any crawling errors ocurred, which might be the case if only some of your URLs get crawled.

share|improve this answer
+1. Additionally, Webmaster Tools will parse sitemap.xml for you, and show you if there are errors. – Jesper Mortensen Oct 16 '09 at 12:27

It can takes weeks for Google to fully index a site. The quality and perceived importance if the page (incoming links) is a contributing factor.

share|improve this answer
  1. Is Google just crawling the first 300, or some other set?

  2. How long have your waited for the crawl? I've seen Google wait a long time (>2 wk) before getting around to crawling everything on a sitemap.

share|improve this answer
This should have been a comment. – Shoban Oct 16 '09 at 7:26

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.