11

Other than virtual memory, How about RAM usage for Varnish?

Is it much RAM needed?

5 Answers 5

17

You can run Varnish in a few different modes. One uses the disk as the cache repository and in this mode, you can safely restart Varnish without losing your cache. This also means you can support a very large cache and let the OS's IO cache keep the most requested content in the server's memory. This mode is configured with the flag -s file,/var/lib/varnish/varnish_storage.bin,1G

The other mode is memory-only - Varnish keeps all cached data only in memory, never writing to disk. This can deliver extremely fast response times, but the cache size is limited to how much memory your server has. Also, if you restart Varnish you lose the cache and suddenly requests will start flooding your backend. Note, however, that Varnish is extremely stable and we only ever restart Varnish when an update is released (every 6 months or so). This mode is configured like -s malloc,3200M

So to answer your question, Varnish performs much better with more RAM. We have a Amazon EC2 instance with 3.5GB RAM and no swap and give Varnish 3GB of RAM for caching. Currently we have almost the entire backend site in cache, including images and PDF files etc. I can definitely recommend this setup.

0
5

http://varnish-cache.org/wiki/FAQ#HowdoIregulatehowmuchmemoryVarnishwilluseforcaching

Which redirects you to here: http://varnish-cache.org/wiki/ArchitectNotes

1
  • 1
    +1 for link to interesting article
    – sumar
    Mar 2, 2010 at 21:43
5

Varnish actually supports multiple storage backends. You can cache something to RAM, whereas something to disk:

-s default=malloc,256m \
-s static=file,/var/lib/varnish/varnish_storage.bin,1G"

This is most useful to conditionally store static files cache (.css, .js files) to disk, and storing full page cache into RAM. You will need to adjust your .vcl file appropriately to cache conditionally into each storage backend: https://www.getpagespeed.com/server-setup/varnish-static-files-cache

This will allow you to keep memory footprint at minimum, whereas still using part of RAM for cache.

4

Caches obviously thrive on memory. But how much you need depends on your site.

Benchmark varnish+your site using request logs, and see how fast memory use grows. Use a vm to test how many requests you can handle with different ram configurations (enable a ram balloon driver).

0

In most situations (exceptions in the elaboration), For good performance you will need free memory on the system (available for the kernel to use to cache the disk) equivalent or larger than the size of the frequently accessed parts of your website, the kernel should be able to use RAM to cache your varnish disk cache, this is probably the reason why disk cache is default in varnish. more free ram than that would probably yield a diminishing return and will not provide the value for money you spend on RAM.

I have appended a modified previous answer below, it did not contain a direct answer to your question, but might help you form an understanding of the relevance of the answer above, i hope it helps.

The default caching in Varnish is Disk not RAM, this is probably because, on a machine dedicated to varnish, the varnish cache on the disk will be cached in RAM anyways (default behavior of the linux kernel), this happens as soon as parts of the file system are read or written.

This also means that having more free ram than the size of frequently accessed parts of the website will improve the performance of varnish greatly, as if it were running with ram cache, while not forcing the server to SWAP (down to it's knees) when another event happens, like an operating system update, or a cron job is fired.

So when should i use dedicated varnish RAM cache ?

If the situations below are likely, then dedicating ram to varnish can be a good idea.

1- Your varnish install is on a virtual machine using some para-virtualization technologie (PARA being keyword as this happens when kernel is shared).

this is because kernel disk cache is handled by the virtualization host, so on a highly loaded machine, that RAM cache could be getting blown every few seconds by a different virtual machine !

2- Having a database (or any other application that reads or writes much to the disk often) with many, frequently accessed, large objects (such as big BLOBS)

such an application on the same machine as varnish will also blow that kernel disk cache for both varnish and the database engine / other application.

I hope this helps you make choices, and i apologize to the moderator for the way my previous post was written.

You must log in to answer this question.