0

I'm trying to prevent the search engines from crawling through SSL version of the site, to prevent content duplication and canonization issues.

I found the following great article: http://www.seoworkers.com/seo-articles-tutorials/robots-and-https.html

Problem is, that I'm using lighttpd, which doesn't seem to have the RewriteCond directive, to limit the rewriting only to SSL.

Can anyone advice if this possible on lighttpd, and give a snippet?

Thanks in advance!

2 Answers 2

1

try this:

$SERVER["socket"] == "a.b.c.d:443" {
  ...

  $HTTP["host"] =~ "<yourhostname>" {
    url.rewrite-once = ("^robots.txt$" => "robots_ssl.txt")
  }
}

the url.rewrite-once should definitely be in your $SERVER["socket"] section of your ssl part to be only applied when your ssl host is accessed.

url.rewirte-once can be placed without conditional but it seems that it will not work all the time according to some forums.

2
  • Hi. Thanks for the tip, but unfortunately it didn't work. I still getting the old robots.txt. Any idea?
    – SyRenity
    Feb 10, 2010 at 18:38
  • dump questions first ;) have you restarted? when yes, try to vary the regex: url.rewrite-once = ("^/robots.txt$" => "/robots_ssl.txt")
    – Christian
    Feb 11, 2010 at 5:53
1

If you have at least version 1.4.19 you should be able to use conditional configuration, with something like:

$HTTP["scheme"] == "https" {
     url.rewrite-once = ("^robots.txt$" => "robots_ssl.txt")
}
1
  • Hi. Just tried this - doesn't work, still getting the original robots.txt page. I placed this snippet outside the $SERVER["socket"] block. I use lighttpd 1.4.22, so this shouldn't be a problem. Any suggestions?
    – SyRenity
    Jan 31, 2010 at 18:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .