We have a XAMPP Apache development web server setup with virtual hosts and want to stop serps from crawling all our sites. This is easily done with a robots.txt file. However, we'd rather not include a disallow robots.txt in every vhost and then have to remove it when we went live with the site on another server.
Is there a way with an apache config file to rewrite all requests to robots.txt on all vhosts to a single robots.txt file?
If so, could you give me an example? I think it would be something like this:
RewriteEngine On RewriteRule .*robots\.txt$ C:\xampp\vhosts\override-robots.txt [L]