Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a site with dynamic pages on them (e.g. record.aspx?id=657). These get added to occasionally. Each of these pages can be found by following various tags, and there's a page (ip restricted so only the google mini can see it) that list all the records as links.

The mini is set up to do a full crawl every night at 3am.

Some of the most recent additions to the database don't seem to be showing up, they're a couple of weeks old now, and even show up in Google's main index.

Any ideas why the new records arn't being indexed?

share|improve this question
add comment

4 Answers 4

up vote 1 down vote accepted
+100

Try this:

  • Check if you're not exceeding the license and capacity of the device (Status and Reports > Crawl Status Page).
  • Check the values of "URLs Found That Match Crawl Patterns" and "Total Documents Being Served" . The difference between them must be small.
  • Check the crawl information of the domain (Status and Reports >Crawl Diagnostics) to see if all the pages are being indexed.
  • If you have access to the log files of your web server, force a recrawl of some page not indexed and see what is the response of the web server (maybe a 404?).
  • After all the tests above, if all the pages are being indexed but not showed in the results, I'd recommend to upgrade the software of the Google Mini. It happened to me some time ago, and the upgrade resolved the issue.
share|improve this answer
add comment

So, just a thought, but have you exceeded the capacity of the device?

share|improve this answer
add comment

From the homepage of the administration of your Google Mini:

  1. Click on "Status and Reports" in the left hand column.
  2. From the drop-down, click on "Crawl Diagnostics".

From here you'll have 4 columns, Host Name, Crawled URLs, Retrieval Errors, and Excluded URLs. The values in these columns are hyperlinks to additional information regarding each column. If there are errors with these particular docs, you'll find your answer here.

share|improve this answer
add comment

Is there anything about that in the Mini's logs? and if I remember, there's a place you can check if the mini can access a file or not. lso make sure that the concerned filetypes aren't being ignored from the config.

share|improve this answer
    
The file types definitely aren't being ignored because there are plenty of them already in the index. Can you remember exactly where you check to see if the mini can access a file? –  Kirschstein Sep 16 '09 at 10:34
    
gewglemini:8443/… You could also use gewglemini:8443/EnterpriseController?actionType=networkSettings to check out if your gmini can reach that page! Hope this helps. –  vn. Sep 17 '09 at 19:41
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.