CDN: Synchronizing cache across edge machines

Question

I built a cheap CDN for my static website based on nginx’s HTTP caching.

Context. Here’s are some parts of my cache configuration:

proxy_cache_key "$scheme://$host$uri"; 
proxy_cache_valid 200 301 302 1200d; 

I have one origin machine and around 10 edge machines all proxying back to the origin at various locations. The origin also doubles up as one edge. Cache on all of these machines are warmed up.

Now, every now and then content in a particular URL will be updated. With a sticky session in a load balanced environment this means that cache on one edge and the origin will be invalidated using proxy_cache_bypass mechanic (I updated content from the admin section and visit the URL. Nginx does a proxy BYPASS refreshing the content because I have a set cookie.)

The problem here is that in all the other edge locations the cache validity for that particular URL is a max of 1200 days. Which means they’ll never fetch from the origin.

In a scenario like this, how I ensure the cache is updated on all my edge servers?

One approach suggested here is to do something like this:

Loop through all the edges and run

curl -o /dev/null -k -I --resolve cdn.yourdomain.com:80:127.0.0.1 https://cdn.yourdomain.com/img/logo.png 

Is there a right approach to this?

Answer 1

I think it would be easier to simply set reasonable Cache-Control: max-age=... headers on the origin responses, than to manually force the caches to refresh certain content. Then, on the nginx caches, if you have proxy_cache_revalidate on on, the caches will only check the origin for particular content once every max-age period. If the content on the origin hasn't change, then nginx will continue to serve from it's cache. However, if the origin content changes, nginx will retrieve and store the fresh copy, and start the process all over again, checking again when max-age expires next.

Answer 2

1200 days a lot.

I don't really see how that curl command helps to update the cache ( if the url already exists in the cache )

You need to purge somehow the cache on edges.

Commercial subscription has a proxy_cache_purge method ( http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_purge ), but i guess you don't have that.

A free alternative, an old module from Frickle, ngx_cache_purge ( https://github.com/FRiCKLE/ngx_cache_purge ) , but the truth to be told , i didn't used with recent Nginx versions. A quick google search suggests that still works. You could have something like this in every edge config :

        location ~ /purge(/.*) {
            allow              1.2.3.4;
            deny               all;
            proxy_cache_purge  tmpcache $1$is_args$args;
        }

Then from 1.2.3.4 you could execute something like curl -H "Host: yourdomain.com" http://5.6.7.8/purge/path/to/file.jpg" , where 5.6.7.8 is the first edge IP ( this would delete /path/to/file.jpg on that edge ) . Then you should loop through all the other edge servers ip's