Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Just a quick question: Can Varnish cache S3 objects? Or it only works for static files on the server?

share|improve this question
    
Why would you want to? Serve them directly off S3 or the CloudFront CDN. Save yourself bandwidth and effort. –  ceejayoz Oct 10 '12 at 13:55

1 Answer 1

Varnish will cache files that are served through it (provided it is setup to do so).

A typical setup will have a backend server provide content, and Varnish setup as a caching layer in front of the server. In this way, Varnish can be configured to cache both static and dynamic content.

In the case of S3, it varies. If you serve your content directly from S3 (i.e. the URL for your resource points to S3) then Varnish is not involved and will not cache the content.

On the other hand, if your server accesses the S3 files locally (e.g. your application reads the file or you use something such as s3fs to mount an S3 bucket locally) and serves the files from it, then Varnish can be used to cache the files.

Arguably though, the question of purpose comes to mind. Serving from S3 should be much more scalable than attempting to implement your own caching using Varnish. The one advantage might be saving a DNS lookup (so possibly good for a small number of small items). (A better caching alternative might be Cloudfront, but for few/small files S3 often performs better).

share|improve this answer
    
Thanks for your answer! The reason i want to cache the S3 items is because I notice some lag when loading a lot of S3 images (on a single page), and after all the images would be requested repeatedly. I am actually pointing to S3 via cname-sub domain, so I guess there is no cache? Is there a way to, as you said, for the application to read the file and hence enable the cache? –  StCee Oct 10 '12 at 14:32
    
Caching server side in this scenario should not appreciably impact the lag client side in this scenario. You are likely limited by the number of simultaneous connections a browser will make if you have a lot of images. A design that implemented CSS sprites and reduced the number of requests would be a good solution, or possibly using multiple static sub-domains to increase parallel fetching of assets (not as good a solution). Caching tends to increase how fast your server can deliver the file, but factors like DNS lookups and round-trip times may have more of an impact in many cases. –  cyberx86 Oct 10 '12 at 14:37

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.