Windows Server 2008 R2 Metafile RAM Usage

Question

I have a server which runs Windows Server 2008 R2 x64 with 4GB of RAM which hosts around 2-3 million files, the majority of which are image files.

Over a course of a week, I have noticed that applications on the server were slowing to a crawl due to excessive paging to the disk due to low memory, which has a knock-on effect to all services currently running on it, causing a major performance issue.

Upon investigation in Task Manager, I noticed that almost all 4GB was in-use but when you look in the Processes tab, the sum of all the memory usage there do not add up and at most only 1.5GB is supposed to be in use.

Using Google to find a solution, it appears that most of the RAM was used in the "Metafile" which is a cache of NTFS information for files on the file system so that the system does not have to query the MFT for information again. This cache is never cleared or marked as "cache" in Task Manager or as "Standby" in Sysinternal's RamMap.

There was a suggestion to install the KB979149 hotfix but upon trying to install it, it says "This update is not applicable to your computer".

The only temporary fixes I have so far found are:

1. Use RAMmap from Sysinternals to "Empty System Working Set" every 1-3 days which marks the cache as "standby" and "cache" in Task Manager so the RAM can be used by other applications.
2. Reboot the machine, which is undesirable as this server is serving public websites.

At the moment I am having to perform the 2. fix every few days to prevent it reaching bottleneck levels.

Before: (800MB RAM used - other applications cannot use this RAM)

After: (800MB RAM marked as cache - available for other applications)

So my question to you all is: Does any method exist out there to limit the RAM usage of this metafile?

Answer 1

Sorry to be so direct but what about you upgrade the server to an amount of ram that is a little higher than a what workstations have these days? 16gb memroy are freaking cheap. Less expensive than even half a day of your time.

Answer 2

I don't claim to be an expert regarding the internal workings of memory or disk caching in a Windows OS, but I have two observations:

1. If the OS didn't cache the data in memory it would have to read it from disk, which is an exponentially slower storage media then memory, so the performance problem you're seeing now would almost certainly be worse.

2. You're trying to solve the problem by treating a symptom of the problem instead of the cause of the problem. The cause of the problem is almost certainly a lack of sufficient physical RAM and my suggestion would be to address that.

In addition, while the cache may be using 1.5GB of RAM I would wonder what the memory usage is for other processes and services and might the solution be to investigate that usage for potential problems.