Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have one dedicated server for Varnish to serve tens of thousands of static files. This server has 32GB of RAM. These files take up almost all the RAM, the vast majority of them are also used regularly, so I stand now before the problem - what to do. The server dealing effortlessly with the handling of requests, but the number of served files is increasing all the time and at any moment I will face the problem of RAM. Adding additional RAM is not an option, as well as saving part of the data on the hard drive.

I decided to put an additional server with the same parameters. I would like to spread the data between the two servers. Both servers will work with HAProxy.

As the amount of data stored in the cache grows very rapidly, and their rotation, I need a solution that will allow me to automatically distribute the data on both servers. For example:

Varnish1 handles site S1, S3, S5
Varnish2 handles site S2, S4, S6

At a time when it appears page S7, should be automatically appended to the least loaded server. Likewise, when the site will disappear, eg S2, the data should be re-distributed the servers to achieve the most balanced load. The same should also happen during the normal operation of servers, when one site, such as S1 will suddenly consume much more resources than normal, the server should automatically transfer part of the supported data to other servers. I accept the partial replication, ie, when the number of requests such as S1 increases to such an extent that will clog up one server, data should be immediately replicated to another server.

I predict that in the future will increase the amount of data served several times as well as the number of servers, Varnish, so I can not rely on hand-held solutions.

share|improve this question
That list of desired features is.. awfully complex. Seems like a load balancer with a simple URL hash-based request distribution would fit most of your needs, but the dynamic adjustment and transfer of cache data to support it might be very difficult to implement. – Shane Madden Feb 28 '12 at 17:55
So In what other way can I solve this? Distributed shared memory? Memcached/Redis cluster? Memcached cluster would be great, because then I could no longer worry about the amount of memory, and "squeeze" Varnish server for maximum power, but what is the cost? – Galmi Feb 28 '12 at 20:58
If you just want to serve static files, I think you better of with multiple nginx server behind load balancer. – Rianto Wahyudi Mar 2 '12 at 2:45
2 low spec nginx server with 64 GB SSD probably gives you better throughput at lower price porint than 1 server with 32GB ram. Static data can be replicated using DRBD or lysncd. – Rianto Wahyudi Mar 2 '12 at 2:52

I'm with Shane Madden. This is a tough one. Here are some thoughts:

Reconsider Disks

Reconsider using disks for Varnish cache. Adding disk can be easier than adding RAM; the faster disks you can afford the better. There will be a drop in performance, but to the average user, it might not be noticeable. We've used disks with Varnish quite successfully. Another advantage of disks: If you go the route of a NAS, for example, you could mount that NAS on different servers so you don't have to worry about synching files across various servers.


With a Proxy, you can divert traffic, but at the end of the day, you will run into the same problem -- running out of RAM or Disk space.

Look Away From Varnish

It might be time to look at a solution other than Varnish. Maybe a CDN? Which may or may not meet your needs, but you could unload some of the heavily used files onto a CDN and for the less used, deliver from your servers.

HTH and good luck!

share|improve this answer

The first thing to do is to check if you handle the Vary HTTP header correctly. This is extremely important because if it is not handled correctly you can get multiple copies of the same cache object which are only separate copies because of an incorrect HTTP header.

What do you get when you run curl -I ?

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.