Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

First off, I use: APC, W3TC, PHP5, Wordpress 3.8, Apache 2.2 and am getting a lot of "Cache Misses"

Still don't really understand Varnish that well. Here are some stats I got from the Unixy backend from a few minutes running:

client_conn: 3744 client_drop: 0 client_req: 3910 cache_hit: 914 cache_miss: 2347

I changed my Varnish memory to 3GB of RAM (out of 8 total).

Is it normal that Varnish only caches pages that are hit frequently? So let's say you have a site with 10,000 pages, is Varnish capable of keeping all those in the cache or does it only cache a certain amount of frequently requested pages?

So my question really is how I can improve Varnish to cache more pages and keep them in the cache for a long time (mostly static pages that dont need to be refreshed a lot)

share|improve this question
1  
Varnish only caches pages by default that have been hit more than $x time in $y minutes, doesn't cache pages with cookies and obviously can only cache pages that have already been requested. –  Mark Henderson Feb 12 at 10:41

2 Answers 2

up vote 1 down vote accepted

3GB should be enough, seriously. If you actually hit the limit, you won't have cache miss but cache drop which is, in your case: 0.

So the memory is probably not the problem.

The varnish memory usage is pretty straight forward, it uses not much memory to work, and most of the memory is used to store objects, object is about your webpages, so an html page of 50kb cached in varnish will use about ~50kb in varnish.

HOWEVER, if your object and therefore its associated object hash is different, for example because there are different query parameters (not impacting the contact but used to track for example), therefore each object will be cached separately.

here is the default vcl to compute hash for object:

sub vcl_hash {
  hash_data(req.url);
  if (req.http.host) {
    hash_data(req.http.host);
  } else {
    hash_data(server.ip);
  }

  return (hash);
}

As you can see it is based on the full url and the host. It exists a querystring-vmod to sort and/or removes query params to increase the hit count.

This may be a good start to check if your object are not considered different by varnish (in most case it is because of query params).

But in your case I actually suspect that you've cookies, and/or wrong cache control headers.

By default, varnish will ignore all requests with cookies, increasing your cache miss count.

sub vcl_recv {
  // ...

  if (req.http.Authorization || req.http.Cookie) {
    /* Not cacheable by default */

    return (pass);
  }
}

If you actually know what you're doing you may remove cookies from requests and force cache of your object but be careful, you may cache an admin page this way which will be delivered to guests.

For a example, you may define which url should be cached and set a Cache-Control: public header, and in your vcl, detect a public Cache-Control header, remove cookies, and cache.

But again, be careful.


EDIT: You may be interested in this article: Varnish and Wordpress on the Varnish docs

share|improve this answer
    
Thank you for the thorough answer, my cache hit rate is about 50% now cache_hit: 13505 cache_miss: 21895 --- I removed some javascript that added cookies just to make sure varnish is working properly. I will now inspect cache control headers –  mashup Feb 12 at 10:52
    
The headers I set via htaccess are "Cache-Control: max-age=3600, public" and Pragma: public –  mashup Feb 12 at 11:01

On top of the Unixy Varnish config I added this after researching various configurations, maybe some of it is redundant but my cache HIT ratio is a lot better now like 60% and I managed to get the CPU load from 3 down to 0.10 - 0.50

wordpress backend

if (!(req.url ~ "wp-(login|admin)")) {
                unset req.http.cookie;
        }

other files

  if ( req.url ~ "(?i)\.(png|gif|jpeg|jpg|ico|swf|css|js|html|htm)(\?[a-z0-9]+)?$" ) {

        unset req.http.cookie;
  }

cookies

if (req.http.Cookie) {
    set req.http.Cookie = regsuball(req.http.Cookie, "(^|; ) *__utm.=[^;]+;? *", "\1");

    if (req.http.Cookie == "") {
        remove req.http.Cookie;
    }
}

purging

if (req.request == "PURGE") {
return (lookup);
}

Also 3GB is a little overkill, I may change it back 1 or 2GB .. even with 10,000 pages or more, but I will let it run for 24 hour and see

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.