0

I am having a few issues with my VPS. Something happened on it which my hosts are unsure of and so I am having to go through an ordeal to try and reconfigure a few things. One of those things are Varnish... I posted a question here: https://webmasters.stackexchange.com/questions/89506/http-header-if-not-modified-help/89507?noredirect=1#comment109928_89507 whereby I am using WP Super cache to create static HTML files of pages using Mod_rewrite to reduce server load. I am then putting these static pages through Varnish to further reduce load.

The issue I'm having is that WP Super cache is sending a max-age=3 which obviously, after 3 seconds is recorded as a miss in Varnish for another 3 seconds. So really, pointless using Varnish.

However, if setting the max-age of the content to being longer, means that if I change a CSS or dynamic page in WordPress, the content is stale in the browser which obviously I don't want.

I was wondering (correct me if this isn't the right way to go) if that by following this guide: https://www.varnish-cache.org/trac/wiki/VCLExampleLongerCaching Varnish can strip out the headers sent by .htaccess in WordPress, then Varnish caches for a week (unless purged either via SSH or via the WP HTTP purge plugin, I guess) and the browser cache is kept low at 15 minutes so if anything changes, it'll only be 15 minutes out of date but after that 15 minutes, the next request will still come from Varnish, not from Apache.

Some of the websites I look after are photography websites so I really don't want to cache GBs of images in Varnish so I have currently told my VCL not to cache these. I only want to cache the pages so I can then get rid of extra caching plugins used by WordPress.

I have tried to see if there was a way to have a long Varnish cache time then WordPress would send a header that it would store in browser for a max-age of say 1 day but if the content is updated (a web page or CSS file for example) then this would be updated in the browser. I haven't seem to have found an answer for this so maybe I'm asking too much.

Here is my VCL:

backend default {
  .host = "public IP";
  .port = "8080";
}

acl purge { "localhost"; "127.0.0.1";}

sub vcl_recv {
    if (req.request == "PURGE") {
    if (!client.ip ~ purge) {
    error 405 "Not allowed.";
    }
    return (lookup);
    }

#set req.grace = 60m;
if (req.request != "GET" && req.request != "HEAD" && req.request != "PUT" && req.request != "POST" && req.request != "TRACE" && req.request != "OPTIONS" && req.request != "DELETE") { 
 return (pipe); } 
if (req.request != "GET" && req.request != "HEAD") { 
 return (pass); } 
#if (req.http.Authorization || req.http.Cookie) { 
#return (pass); } 
return (lookup); 
# Set X-Forwarded-For header for logging in nginx
remove req.http.X-Forwarded-For;
set req.http.X-Forwarded-For = client.ip;

# Remove has_js and CloudFlare/Google Analytics __* cookies and statcounter is_unique
set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(_[_a-z]+|has_js|is_unique)=[^;]*", "");
# Remove a ";" prefix, if present.
set req.http.Cookie = regsub(req.http.Cookie, "^;\s*", "");

# Either the admin pages or the login
if (req.url ~ "/wp-(login|admin|cron|cart|my-account|checkout|addons|administrator)") {
# Don't cache, pass to backend
return (pass);
}
if (req.url ~ "/administrator") {
  return (pass);
} 
if ( req.url ~ "\?add-to-cart=" ) {
 return (pass);
}
if (req.url ~ "/(contact-us|contact|get-a-quote|upload-files|competition)")
{
return(pass);
}
# Never cache PUT, PATCH, DELETE or POST requests 
#if (req.method == "PUT" || req.method == "PATCH" || req.method == "DELETE" || req.method == "POST") { 
#return (pass);
#} 
# Remove the wp-settings-1 cookie
set req.http.Cookie = regsuball(req.http.Cookie, "wp-settings-1=[^;]+(; )?", "");

# Remove the wp-settings-time-1 cookie
set req.http.Cookie = regsuball(req.http.Cookie, 
"wp-settings-time-1=[^;]+(; )?", "");

# Remove the wp test cookie
set req.http.Cookie = regsuball(req.http.Cookie, 
"wordpress_test_cookie=[^;]+(;)?", "");

# Static content unique to the theme can be cached (so no user uploaded images)
# The reason I don't take the wp-content/uploads is because of cache size on bigger blogs
# that would fill up with all those files getting pushed into cache
if (req.url ~ "lib/themes/" && req.url ~ 
"\.(css|js|png|gif|jp(e)?g)") {
unset req.http.cookie;
}

# Even if no cookies are present, I don't want my "uploads" to be cached due to their potential size
if (req.url ~ "/lib/uploads/") {
return (pass);
}

# any pages with captchas need to be excluded
if (req.url ~ "^/contact/" || req.url ~ "^/links/domains-for-sale/")
{
return(pass);
}

# Check the cookies for wordpress-specific items
if (req.http.Cookie ~ "wordpress_" || req.http.Cookie ~ "comment_") {
# A wordpress specific cookie has been set
return (pass);
}

# allow PURGE from localhost
if (req.request == "PURGE") {
if (!client.ip ~ purge) {
error 405 "Not allowed.";
}
return (lookup);
}

# Force lookup if the request is a no-cache request from the client
if (req.http.Cache-Control ~ "no-cache") {
return (pass);
}

# Try a cache-lookup
return (lookup);

}

sub vcl_fetch {
#set obj.grace = 5m;
#set beresp.grace = 60m;

}

sub vcl_hit {
if (req.request == "PURGE") {
purge;
error 200 "Purged.";
}
}

sub vcl_miss {
if (req.request == "PURGE") {
purge;
error 200 "Purged.";
}
}


sub vcl_deliver {
# multi-server webfarm? set a variable here so you can check
# the headers to see which frontend served the request
#   set resp.http.X-Server = "server-01";
   if (obj.hits > 0) {
     set resp.http.X-Cache = "HIT";
   } else {
     set resp.http.X-Cache = "MISS";
   }
}

So currently, varnish purges for 5 minutes but my max-age in .htaccess on the sites are 3 seconds which is just pointless.

Am I thinking in the right direction? Hopefully I've made sense. Hopefully someone can show me the light!

Thanks!

  • further to my post above, I tried implementing the code from Varnish-Cache for longer caching which seems to give the desired effect. However, all it does it give a max-age of 900 to the browser (which is okay I guess) but still the first initial visit of the page is reported as a miss in Varnish despite my visiting that specific page on another devise so in theory it should already been in the cache…Can anyone offer any advise? – WPDEVE Feb 3 '16 at 10:28
1

Since you're using Varnish, I would suggest taking out your custom .htaccess caching headers on the backend and control the caching headers from within Varnish. That VCLExampleLongerCaching example on the Varnish site should do this correctly for you.

I find that curl is very useful for making test requests to websites to see what caching headers come back. You can do this to see just the headers returned for a given request:

curl -s -D - http://www.example.com -o /dev/null

I have tried to see if there was a way to have a long Varnish cache time then WordPress would send a header that it would store in browser for a max-age of say 1 day but if the content is updated (a web page or CSS file for example) then this would be updated in the browser.

This is a very common requirement and how many sites handle it is by adding a unique hash to the end of each CSS/JS file's URL in the page source. Each time the file changes, the unique hash changes too, so that asset now has a new, uncached URL. So you just need to clear the HTML page from Varnish and any changes to referenced assets will get updated automatically. It appears that Wordpress uses this technique too - I just had a look at a random Wordpress site and see that date string like "?v=20150727" is at the end of each CSS file in the source.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.