Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

When staging a website on a remote VPS, we would like it to be accessible to team members only, and we would also like to keep the search engine bots off until the site is finalized.

Access control by host whether in Iptables or Apache is not desirable, as accessing hosts can vary.

After some reading in Apache config and other SF postings, I settled on the following design that relies on restricting access to only through specific domain names:

Default virtual host would be disabled in Apache config as follows - relying on Apache behavior to use first virtual host for site default:

<VirtualHost *:80>
# Anything matching this should be silently ignored.
</VirtualHost>

<VirtualHost *:80>
    ServerName secretsiteone.com
    DocumentRoot /var/www/secretsiteone.com
</VirtualHost>

<VirtualHost *:80>
    ServerName secretsitetwo.com
    ...
</VirtualHost>

Then each team member can add the domain names in their local /etc/hosts:

xx.xx.xx.xx   secrethostone.com      

My question is: is the above technique good enough to achieve the above said goals esp restricting SE bots, or is it possible that bots would work around that.

Note: I understand that mod_rewrite rules con be used to achieve a similar effect as discussed here: How to disable default VirtualHost in apache2?, so the same question would apply to that technique too.

Also please note: the content is not highly secretive - the idea is not to devise something that is hack proof, so we are not concerned about traffic interception or the like. The idea is to keep competitors and casual surfers from viewing the content before it's released, and to prevent SE bots from indexing it.

share|improve this question
add comment

1 Answer

up vote 3 down vote accepted

Sure, if you choose a very random domain name, it should be fairly well hidden. However, it would probably be just as easy (and wouldn't require everyone to modify their hosts file) to just throw it behind an http digest authentication prompt. Then just distribute the credentials to your team members, and they'll be able to access it.

In your VirtualHost, just specify a location to protect. This will likely be the same path as your VirtualHost DocumentRoot:

<Location /var/www/secretsiteone.com>
    AuthName 'Nothing to see here'

    AuthType Digest
    AuthDigestProvider file
    AuthDigestDomain /
    AuthUserFile /path/to/.htdigest

    Require valid-user
</Location>

Then generate that .htdigest file:

htdigest -c /path/to/.htdigest 'Nothing to see here' 'username'
share|improve this answer
    
I was going to suggest the same thing, that is some form of simple auth protection. –  WerkkreW Apr 6 '12 at 7:25
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.