8

I'm developing a caching system for an ecommerce platform that will use a reverse proxy for caching. I plan to handle invalidation by using proper HTTP/1.1 headers. That is, I will set an ETag on first generation of the content and cache that ETag value in the application. The Cache-Control header will specify "must-revalidate" so the proxy should set If-None-Match header on subsequent requests with the ETag. The application will lookup the cached ETag value and if it matches it will send a 304 response, otherwise it will generate a full 200 response.

I hoped to use nginx but I can't tell for sure that it supports ETags (docs indicate it doesn't but maybe they are out of date?). Varnish is another option but I'm not positive here either..

Which reverse proxy servers out there have full support for ETags? I'd like it to actually cache multiple versions so I can do things like split testing without having to disable the cache. That is, HTTP/1.1 specifies that a client can send If-None-Match with multiple ETag values and the server should respond with which ETag matched (if any). If the reverse proxy kept multiple copies rather than just the last-seen value and let the server specify on each request which to use, that would be ideal.

5 Answers 5

2

I just checked in Varnish source code and even though it support If-Modified-Since and If-None-Match headers, it does not support must-revalidate in Cache-Control. The only supported attributes in Cache-Control are max-age and s-max-age.

References:

2
  • Thanks. I don't know much about Varnish VCL, but is it possible to specify using VCL to always revalidate given that Varnish does seem to support revalidation?
    – ColinM
    Mar 15, 2012 at 0:03
  • Just found the "experimental-ims" branch of Varnish seems to add complete support for If-None-Match. Hopefully it will eventually be merged into a stable release. varnish-cache.org/trac/wiki/BackendConditionalRequests
    – ColinM
    Dec 21, 2012 at 6:21
2

nginx requires third party modules to support ETag. And there are two of them.

2
  • The static etags module is for generating etags, not caching content with etags. Similarly, the dynamic etags module generates etags for dynamic content for upstream use, not reverse proxy use. However, I think nginx 1.3 added official support for etags with reverse proxies..
    – ColinM
    Oct 24, 2012 at 20:08
  • 1
    Just for future reference, nginx supports ETag and If-None-Match as of 1.7.3, but it can still only cache one result for a given url so I wouldn't call this full support. That is, you can't have multiple responses cached and negotiated with the server during revalidation. See trac.nginx.org/nginx/ticket/101
    – ColinM
    Jul 6, 2016 at 15:25
1

Correct me if I'm wrong, and I know this is an old post - but I'd like to comment for new passers-by. I believe a Reverse Proxy cache doesn't help as much as you'd like when using ETags.

Validation caching mechanisms use the origin server to validate if the ETag (or last-modified date) in the request is still valid (matches or doesn't match the resources etag, depending on which header is used, or has/has not been modified since date given in request).

This means a reverse proxy cache such as Varnish will still pass that request through to the origin server. It may respond with the request rather than have the server handle it, but you didn't save the round trip to the origin server.

Browsers can cache responses and handle a 304 response in any case, so the user's private cache may be better suited to handle this than using a reverse proxy (YMMV, especially at scale, and depending on your use case of course. I don't want to make assumptions about your apps).

From the spec 13.3:

When a cache has a stale entry that it would like to use as a response to a client's request, it first has to check with the origin server (or possibly an intermediate cache with a fresh response) to see if its cached entry is still usable. We call this "validating" the cache entry. Since we do not want to have to pay the overhead of retransmitting the full response if the cached entry is good, and we do not want to pay the overhead of an extra round trip if the cached entry is invalid, the HTTP/1.1 protocol supports the use of conditional methods.

and then note 13.3.4:

An HTTP/1.1 caching proxy, upon receiving a conditional request that includes both a Last-Modified date and one or more entity tags as cache validators, MUST NOT return a locally cached response to the client unless that cached response is consistent with all of the conditional header fields in the request.

So, Varnish can return a response for you, but you still have a round-trip to the server. If you can use a app-cache such as APC or memcache, then that still might be worth it to you. Validation caching is generally better for bandwidth savings over server-resource savings, however.

Validation caching might best be left to the client (browser or api code).

Using the Expiration model for caching is where a reverse-proxy cache really shines. This lets you skip hitting the origin server altogether. Using Expires, Cache-Control, Date, etc, is the best (again, IMO) mechanism for a reverse proxy cache as the cache can return the response, assuming its not stale, without ever hitting the origin server.

1
  • The use case I had in mind for this question is to have the proxy revalidate with the origin server and the origin server return 304 if the ETag matches the ETag that would be cached for a given record. That is, the ETag would be randomly generated when the page is first rendered and saved with the primary key for that record. If the record is modified the ETag is erased.
    – ColinM
    Mar 26, 2014 at 15:00
1

You can look at Apache TrafficServer, which seems to have what you need.

1
  • Docs are very sparse on ETag support details but thanks for mentioning TrafficServer!
    – ColinM
    Oct 24, 2012 at 20:04
0

To date I believe there are still no proxies that fully support this HTTP spec. So, about a year ago I decided to write my own using Node.js and MongoDb.

https://github.com/colinmollenhour/node-caching-proxy

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .