1

Is there a way to swap out a robots.txt file in nginx based on hostname? I currently have www.domain.com and backup.domain.com pointing at the same nginx server, but I don't want Google indexing backup.domain.com.

2

Two options:

  1. An if statement based on $http_host that rewrites to one of two text files on the backend.
  2. Have robots.txt be rewritten to a dynamic script (robots.php etc.) that responds accordingly.
1

There is a proper way with map

map $http_host $examplecom_robotstxt {
  hostnames;
  default robotstxt/development.txt;
  example.com robotstxt/production.txt;
}

server {
  ...
  location = /robots.txt {
    rewrite .* /$examplecom_robotstxt break;
  }
  ...
}

Please note that map definition should be outside of server definition.

0

For reference, this is the syntax I used:

if ($http_host != "www.domain.com") {
    rewrite ^/robots.txt /robots-backup.txt last;
}

This was inside the location block

  • if in location is evil. don't use it. never. – cadmi May 2 '13 at 4:02
  • Can you suggest a better solution? – Noodles May 2 '13 at 9:35
  • two server {} sections, of course. it's right way, recommended by nginx author. – cadmi May 5 '13 at 8:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.