I need to serve a large number of very small static files (e.g. 1Million, ~1KB), from a VPS. My current favoured approach is Nginx, but my main concern is hitting slow VPS disks.

Is there an approach I can use which would serve these files from local memory, rather than going to the Disk? I understand that the OS (Debian in this case) will keep 'hot' files in memory, but I'm worried that this will be difficult to monitor or influence. Perhaps there is a way to get Nginx to serve from memory? Or some other approach entirely?

  • This seems to be a duplicate of serverfault.com/questions/397378/… – Uwe L. Korn Oct 30 '12 at 9:24
  • Well that's pretty close, but its really a badly titled question (it's about a specific error he has, not nginx/ram in general). I'm also open to non nginx approaches. – UpTheCreek Oct 30 '12 at 9:26

You can put the folder they are in in the memory by mounting it as a tmpfs partition.

For example: mount -t tmpfs -o size=128m tmpfs /mnt/tmp

| improve this answer | |
  • Quick question - do you know if this starts copying the whole folder into memory as soon as you mount this, or does it load each file into memory the first time it's requested? – UpTheCreek Oct 30 '12 at 11:17
  • The mount is a reserved section of memory. So you copy the data to the mount and once copied, they will reside in memory. Note that this folder is not persistent and won't last across reboots. – ewwhite Oct 30 '12 at 13:55

I hate to be contrary, but don't do this. Assuming you're using Linux, the OS will in any case cache frequently-accessed files in memory that is not more urgently needed by the OS. If you try to second-guess this process, eg by using a ramdisk as Jeroen suggests, you'll either be right or wrong about your files needing to be in memory more than what the kernel might otherwise keep there.

If you're right, then all you've done is what the VM system was going to do for you anyway.

If you're wrong, then some other piece of memory that would have displaced your files will instead be flushed to swap, and the retrieval of those pages from swap will put more of a performance load on your server than the retrieval of your files from the FS would have done.

The VM code in Linux is tightly and intensively tuned. It doesn't always get it right, but it tends to get it right such a large fraction of the time that you need a really good reason to override it.

| improve this answer | |
  • Thanks. I just have a hard time believing that the standard built in hot file caching config will work in all scenarios. I mean, will it really keep 1GB of files cached in memory? – UpTheCreek Oct 30 '12 at 10:57
  • But without certain tools, you don't know which files or portions of files are in the VM subsystem's cache. If I read a 1GB file by an application, it's going to end up being cached. What if I don't want it cached? For the OP's need, service web content, I don't think there's harm in taking the ramdisk or vmtouch approach. – ewwhite Oct 30 '12 at 14:02
  • I have to agree with @MadHatter here. This is so rarely an issue with real world web sites that it's not worth bothering, unless you actually find a performance problem traceable to memory caching. Before you do it, see if you actually need to do it. – Michael Hampton Oct 30 '12 at 20:53

I understand that the OS (Debian in this case) will keep 'hot' files in memory, but I'm worried that this will be difficult to monitor or influence.

This is true. You can influence this by tuning your virtual memory options;

Read about swappiness here. Set it to 1 or 0 in your case:

echo 0 > /proc/sys/vm/swappiness
# This will set it permanently, to continue after reboot
sysctl -w vm.swappiness=1

You could make sure your cache pressure is 100, or more;

echo 100 > /proc/sys/vm/vfs_cache_pressure
# Set permanently for after restart
sysctl -w vm.vfs_cache_pressure=100

Read about all these options and more here.

Another option is to place all your files on to a ramdisk; a partition that exists in your VPS's memory. See instructions here and here. Obviously for this option though, be aware that the ramdisk is destroyed at reboot. It will be recreated empty, so at start-up, create and mount a ramdisk, then copy files on to it. Don't make that the only copy of them :)

| improve this answer | |

Ramdisk is a good idea. You can also get rad, and try vmtouch to lock a directory into the filesystem cache. It's more elegant since it can be daemonized and eliminates the issue of non-persistent RAM disks.

Lock all files in a directory into physical memory:
vmtouch -dl /var/www/html/directory-of-small-static-files/

Other examples...

| improve this answer | |
  • That looks great too thanks. I was wondering how to get some visibility into this under the hood magic. – UpTheCreek Oct 30 '12 at 10:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.