Imagine we have an online store site. It contains list of items to sell, and prices associated with it. Due to messy math behind price (they used to offer discounts based on user history, and we can't change this), each page may take time to be rendered.

This is not generally a problem, unless search spiders/bot come to reindex the site aggressively. And site starts to count prices for each bit request despite bots won't have any "personal" discounts etc.

I'd like to use nginx cache feature like but the problem is nginx won't cache answers that have 'Set-Cookie' headers. And yes, site's pages always try to set cookie even for bots.

I can't modify site's code, so the only idea is to use cache and serve staled cache for bots only.

map $http_user_agent $do_not_cache_it {
  "~Bot"  "1";
  default "0";
server  {
  location / {
    proxy_cache_bypass $do_not_cache_it;
    proxy_ignore_headers "Set-Cookie";
    proxy_hide_header "Set-Cookie";

this works but cookies are cleared and hidden for all requests, not bot-initiated only, so even human users can not, say, login to the site.

Any way I can play this nginx cache game successfully, please?


You must log in to answer this question.

Browse other questions tagged .