You are experiencing the Linux Ate My Ram issue.
This is NOT a problem.
Your system is Working As Designed.
The problem is not your OS -- the problem is your understanding of what "free" memory is.
Unix systems use memory for more than just running programs. Memory might be used for:
- Running programs (active/used)
- Buffering data in transit (buffers)
- Caching data recently read from/written to the disk (cache)
- Absolutely nothing (free)
What follows is a brief (and largely incomplete) tour of how modern Unix systems report RAM usage.
What is Free memory (the OS definition)?
When a Unix system is reporting RAM as Free it means "I am not using this RAM for anything".
Free RAM is effectively worthless - It is not making your system faster, it's just sitting there being "free" in case something needs it. That something could be any of the three other items I mentioned above.
What are Cache and Buffer memory?
Cache and Buffer memory are RAM the operating system is using to make your system faster.
This memory is not needed for running programs right now, so your OS is using it to hold data that it needs frequently -- for example the C library (needed by pretty much every program you run) is almost always held in
cache memory, so the system doesn't have to go to the disk to find the instructions it needs to print "Hello World" on the screen.
It's actually a lot more complicated than that -- there's shared memory, wired memory, etc. -- but for our purposes this simple explanation is adequate.
What is Active memory?
Active memory is part of we understand as "used" memory -- RAM that applications are using for whatever it is they do -- sorting spreadsheets, serving web pages, editing graphics, etc.
"Active" memory has been "active" recently -- the program claiming it has made use of its contents (reading or writing), and it's not considered a good candidate for swapping out.
What is Inactive memory?
Like Active memory, Inactive memory is RAM that applications are using for whatever it is they do. The difference is this memory hasn't been accessed in a while, so if push comes to shove the OS thinks it can be swapped out to disk and (with a little luck) the program claiming it won't ask for it again so it will never notice.
What is "Used" memory (the HUMAN definition)
What you and I think of as "Used" memory is, essentially, the sum of Active and Inactive memory. All the RAM currently claimed by applications for their use.
As long as you have more installed RAM than the sum of Active and Inactive memory (plus a nice safety margin of say 512-1024MB on top) you're in an OK place: Your OS probably won't be hitting swap and killing performance.
What is "Free" memory (the HUMAN definition)?
What you and I think of as "free" memory is the memory available to run programs.
This is slightly more complicated than just the "Free" figure your OS reports. When a program asks for RAM the operating system will try to get that RAM in the least disruptive way it can:
- If there is Free memory available (sitting around doing nothing) that RAM will be allocated.
- If there is no Free memory available the OS will cannibalize the Cache and Buffer space: The least-recently/least-frequently accessed stuff in the buffer pool will be tossed out, and that RAM given to the program.
- If there is no Buffer/Cache RAM to cannibalize the swapper will look at the inactive memory and pick the regions it thinks are least likely to be accessed. That data will be paged out to swap (disk), and the newly-freed RAM given to the program.
- If all the Inactive RAM has been swapped out the swapper will start putting Active RAM on disk.
(This is about where performance usually goes to the dogs: Every time a program gets its turn on the CPU its swapped-out bits need to be brought back into RAM, which means some other program's Active memory has to be swapped out -- the high turnover in swap is called thrashing)
- If the system has swapped out everything it can (and filled up the swap partition), or if you're running a system without a swap partition, Bad Things happen. At this point one of two things will occur:
malloc() will fail. This is the POSIX-conforming behavior - the operating system will tell the program asking for RAM that it can't satisfy the request.
The program can either ask for less RAM, or if it can't make do with a smaller chunk of memory it can clean up and exit. (If the program is badly written it will simply crash.)
- If you're on a Linux box, the OOM-Killer may go on a gang-style drive-by killing spree, terminating other processes to try to free up enough RAM to meet the request.
In case you can't tell by my description here and my answer on the linked question, I think this is a terrible way to deal with the problem.
Why does Free RAM go up when you delete files?
In the example from the question here you noticed that it's possible to "Free" RAM by deleting the backup file -- the explanation for that is pretty simple:
Since nothing is using that file (no open file handles) and it's no longer accessible from the filesystem (unlinked) the OS knows nobody will ever access that data again, and it purges the data from the filesystem cache.
This makes the OS report more Free memory, but has no impact on system performance.