Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.


Google crawlers appear to be visiting my site but it is not getting indexed, what am I doing wrong?

Yahoo has managed to find the mydomain.appspot.com and has indexed successfully (albeit on the apspot address and not my domain) so I assume that my HTML is OK.

1) I have set up CNAME records so that www.mydomain.co.uk points to mydomain.appspot.com. When I look at "fetch page as google bot" using web tools the HTML page is being displayed. The crawl stats show activity over the last couple of days.

2) I have web forwarded my mydomain.co.uk to www.mydomain.co.uk, this works a treat apart from when i go to "fetch page as googlebot" i get just the following rather than the HTML page

HTTP/1.1 302 Found
Date: Tue, 29 Jun 2010 21:20:02 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
X-AspNet-Version: 2.0.50727
Location: http://www.mydomain.co.uk
Cache-Control: private
Content-Length: 0

Any suggestions appreciated,

Many thanks

share|improve this question

3 Answers 3

register the domain www.mydomain.co.uk in webmaster tools, set the preferences to use www.mydomain.co.uk, leave mydomain.co.uk in webmaster tools. You'll have to reclaim www.mydomain.co.uk by putting the meta header in.

Google is fetching the mydomain.co.uk which is doing a 302 redirect to www.mydomain.co.uk, rather than fetching www.mydomain.co.uk. Once you adjust webmaster tools, you should see stats in a few days for crawls.

share|improve this answer

Many thanks for this, I had guessed that this was the problem but didnt know how to fix it! The URL preferences were already in place, however, I am being thick about the reclaiming the WWW.mydomain.co.uk in the header data.

Do you mean use a http-equiv="refresh" content="0;URL http://www.mydomain.co.uk" statment to do the redirect?

If so, at present, both the naked domain and the www subdomain are pointing at the same bit of html code. The result is that it loops.

Many thanks

share|improve this answer

In webmaster tools you have to put the:

<meta name="google-site-verification" content="codethatgooglegivesyou" />

in the header of the document.

remove the meta-refresh, that isn't what you need.

Waiting three minutes... Lord this site makes it painful to want to offer help.

share|improve this answer
Hi, this has now worked, google is indexing it, many thanks for your help. The Dashboard has not populated yet on the webmaster tools, but perhaps I have to be more patient? –  user47122 Jul 6 '10 at 19:10

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.