Consider the following scenario:

  • I am running lighttpd-1.4.19 and I am handling big uploads (hundreds of MBs but less than 1GB) through it.
  • Server is running on Ubuntu 8.04 LTS.
  • Files are temporarily written to /var/tmp.
  • PHP5 is handling the end-result through fastcgi.
  • During an upload, upload chunks are written in the temporary directory.

The weird thing is that although the chunks are written on disk, free memory on the server is reducing gradually up to the full file size during the upload. I would understand RAM diminishes by the size of each chunk (let's say 1-2MB) until they're written on disk but I don't understand why it is by the whole file size (let's say 300MB).

  1. Would you know why the RAM would be taken for the whole file although chunks are written on disk? Is it possible /var/tmp is mounted directly in memory? Versus /tmp?
  2. If not, would you know how to diagnose and how to pinpoint which process is taking up the memory? I didn't see the lighttpd nor php-cgi process increasing with ps but maybe I'm not looking at the right thing.
  3. Or better yet, would you have a recommendation on a good web server (Apache, Nginx, lighttpd, etc.) and config to handle big uploads without storing everything in RAM?


  • How are you monitoring RAM usage? – jldugger Jun 1 '09 at 16:38
  • I'm using something like: ps -eo pid,ppid,rss,vsize,pcpu,pmem,cmd -ww --sort=vsize – lpfavreau Jun 1 '09 at 18:16

If the lighttpd or php-cgi are not increasing in size, then don't worry - the increase you're seeing isn't in the RAM consumption of those processes, but is instead in the kernel's use of disk cache. Basically since you just wrote those large files out, the kernel keeps them around in memory on the assumption that you might want to read them again. It will of course drop this cache if that memory is needed by other processes.

| improve this answer | |
  • Thanks for this nice answer. Is there a way to validate this usage with a command? – lpfavreau Jun 1 '09 at 18:18
  • ps will show you %MEM and RSS and VSZ of the lighttpd and php-cgi processes - if they're not increasing, your memory usage is elsewhere. I think one of the lines in /proc/meminfo might tell you how much disk cache the kernel is doing, but I forget which. – pjz Jun 2 '09 at 18:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.