Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top


Google crawlers appear to be visiting my site but it is not getting indexed, what am I doing wrong?

Yahoo has managed to find the and has indexed successfully (albeit on the apspot address and not my domain) so I assume that my HTML is OK.

1) I have set up CNAME records so that points to When I look at "fetch page as google bot" using web tools the HTML page is being displayed. The crawl stats show activity over the last couple of days.

2) I have web forwarded my to, this works a treat apart from when i go to "fetch page as googlebot" i get just the following rather than the HTML page

HTTP/1.1 302 Found
Date: Tue, 29 Jun 2010 21:20:02 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
X-AspNet-Version: 2.0.50727
Cache-Control: private
Content-Length: 0

Any suggestions appreciated,

Many thanks

share|improve this question

register the domain in webmaster tools, set the preferences to use, leave in webmaster tools. You'll have to reclaim by putting the meta header in.

Google is fetching the which is doing a 302 redirect to, rather than fetching Once you adjust webmaster tools, you should see stats in a few days for crawls.

share|improve this answer

Many thanks for this, I had guessed that this was the problem but didnt know how to fix it! The URL preferences were already in place, however, I am being thick about the reclaiming the in the header data.

Do you mean use a http-equiv="refresh" content="0;URL" statment to do the redirect?

If so, at present, both the naked domain and the www subdomain are pointing at the same bit of html code. The result is that it loops.

Many thanks

share|improve this answer

In webmaster tools you have to put the:

<meta name="google-site-verification" content="codethatgooglegivesyou" />

in the header of the document.

remove the meta-refresh, that isn't what you need.

Waiting three minutes... Lord this site makes it painful to want to offer help.

share|improve this answer
Hi, this has now worked, google is indexing it, many thanks for your help. The Dashboard has not populated yet on the webmaster tools, but perhaps I have to be more patient? – user47122 Jul 6 '10 at 19:10

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.