Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a url which i don't want indexed:

So when i google search for "", i don't want the above link to appear in the search results.

However, the following urls are part of public viewing:

Can this be done? Can robots file be used for this?

share|improve this question
up vote 2 down vote accepted

A few things you can do:

Ultimately, though, nothing secret should ever go directly in a URL like that - not all web crawlers obey these robots instructions, and some (spambots etc.) specifically search them out and look at them.

share|improve this answer
Right! Basically, the hidden link is like a preview content before being posted for public viewing (or deleted). Theres no secret or personal data being hidden there. So i thought of this approach to avoid code duplication. You think its fine in this case? – fenderplayer Jan 7 '10 at 20:44

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.