Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have one server that provides access to some hundreds of files. All of them are really big (some have more than 10GB). These files won't change and are read only, but I'll need to provide access to them via a WAN.

I want to make the access faster by using a reverse proxy server running near my users. For example:

  • User X wants to access something on server A.
  • User X access a reverse proxy server on his LAN which causes a cache miss. The file is downloaded to this proxy server.
  • The next time user X wants the same file, he doesn't need to go get it in my main server.

I know that this situation is trivial, but I won't have a lot of traffic, my problem is more related to the file sizes themselves. I read something about Squid having problems caching files larger than 2GB. Does Varnish face the same kind of problems?

Thank you for any thoughts on that.

share|improve this question
add comment

2 Answers

It's technically possible, but for what you want to achieve - I'd consider simply mirror the backend server periodically with rsync.

One of the reasons for this, is how Varnish delivers objects from its cache. If the object is not in Varnishs cache, it will fetch it from the backend. When this happens, the user will have to wait - without any transferring to the client occurring.

When the file is stored in the Varnish cache, transfer to the user can start.

Depending on how the users access the servers in the first place, this can cause significant amount of extra waiting.

share|improve this answer
2  
Varnish 3.0 has support for object streaming, which addresses this. See this Varnish blog entry –  rossnz Oct 25 '11 at 8:56
add comment

Squid needs ./configure --with-large-files for supporting objects large than 2 GB (should be enabled by default on Debian/Ubuntu squid packages).

If most files are accessed often by most clients, consider putting up an rsync mirror instead.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.