Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Our site has a number of large PDF and MP3 files which we would like to cache in Varnish as static files. Currently we don't do much special - simply remove the cookies in vcl_recv and set resetp.ttl = 100w; in vcl_fetch.

The problem seems to be when one of these files is requested (maybe by older browsers) and it's not already in the Varnish cache. There is a delay while Varnish downloads the file from the backend. My understanding is that it doesn't start delivering to the client until the data is fully loaded. This may take 20 seconds or so and sometimes Adobe Acrobat or the MP3 plugin get confused.

Is there a way to both pass the content directly while download and save it in the cache for the next matching client request?

share|improve this question

2 Answers 2

The 3.0.2+streaming branch is not up-to-date anymore and it is only supported for Varnish Plus subscribers, no community support on that.

So if you need streaming capabilities you can either: * Try Varnish Plus (only available to subscribers) which has built in support for it: https://www.varnish-software.com/what-is-varnish-plus * Test and see if the upcoming Varnish Cache 4.0 (release expected in Q2 2014) solves this issue for you (it should).

share|improve this answer

Reading suggestion: https://www.varnish-software.com/blog/http-streaming-varnish

With vanilla Varnish 3.0 you could set *beresp.do_stream* to true in vcl_fetch. The file will be streamed while it is fetched from backend. The caveat: The object is set to busy while streaming, so other clients will be put on hold.

There is also a Varnish 3.0.2 release with baked in "full" streaming support available: http://repo.varnish-cache.org/test/3.0.2+streaming/ which people are using in production, so maybe you want to give it a try.

Or maybe you could "prewarm" your files, placing them in the Varnish memory cache before users start downloading it?

share|improve this answer
    
Thanks - testing out the "beresp.do_stream" options. –  Nicholas Tolley Cottrell Aug 27 '12 at 18:44
    
How do you do prewarming? –  drumfire Oct 6 '12 at 13:54
    
Normally it should be enough to just request the files (e.g. via a script). How long the files are cached in memory depends on your varnish/memory setup. In my old company on patch day we used a script to curl the most visited URLs to have them in cache before bringing the server online again. –  neovatar Oct 10 '12 at 16:08

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.