I have a server currently sitting at 97% memory utilization (Just over 30GB on a 32GB server), yet only show about 1gb of memory being used when I add up the values in Task Manager or Resource Monitor. Any thoughts on how to find out what is consuming all of my memory resources?
There are more memory classifications than are shown in task manager. In particular, the Standby List. RAMMap from SysInternals would be the defacto tool for providing that information.
Active: (also called Valid) The page is part of a working set (either a process working set, a session working set, or a system working set), or it’s not in any working set (for example, nonpaged kernel page) and a valid PTE usually points to it.
Transition: A temporary state for a page that isn’t owned by a working set and isn’t on any paging list. A page is in this state when an I/O to the page is in progress. The PTE is encoded so that collided page faults can be recognized and handled properly. (Note that this use of the term “transition” differs from the use of the word in the section on invalid PTEs; an invalid transition PTE refers to a page on the standby or modified list.)
Standby: The page previously belonged to a working set but was removed (or was prefetched/clustered directly into the standby list). The page wasn’t modified since it was last written to disk. The PTE still refers to the physical page but is marked invalid and in transition.
Modified: The page previously belonged to a working set but was removed. However, the page was modified while it was in use and its current contents haven’t yet been written to disk or remote storage. The PTE still refers to the physical page but is marked invalid and in transition. It must be written to the backing store before the physical page can be reused.
Modified no-write: Same as a modified page, except that the page has been marked so that the memory manager’s modified page writer won’t write it to disk. The cache manager marks pages as modified no-write at the request of file system drivers. For example, NTFS uses this state for pages containing file system metadata so that it can first ensure that transaction log entries are flushed to disk before the pages they are protecting are written to disk.
Free: The page is free but has unspecified dirty data in it. (These pages can’t be given as a user page to a user process without being initialized with zeros, for security reasons.)
Zeroed: The page is free and has been initialized with zeros by the zero page thread (or was determined to already contain zeros).
Rom: The page represents read-only memory.
Bad: The page has generated parity or other hardware errors and can’t be used.
Modern operating systems only make memory free if they have some reason to do so. It takes resources to make memory free and that effort is just wasted when it has to make the memory used again. So they only bother to make memory free if they have absolutely no choice.
The memory is just still being used by whatever used it last. Most likely, it contains copies of data on disk. If that data is read again, it will save the system a disk operation. So this is preferable to having it free.
You're thinking you want memory to be free now so you can use it later. But memory doesn't have to be free now to be used later. In fact, memory that's being used now is more likely to be used later.
Imagine you walk into a factory where all the employees are doing things. Maybe some are doing unimportant things like sweeping the floor or sitting near the telephone in case it rings. But nobody is on the couch doing nothing at all. Do you wonder why more employees aren't sitting on the couch doing nothing -- ready to work should there be something super-important to do? There's always something you can do. Maybe it's not the most useful thing in the world, but it's better than sitting on the couch doing absolutely nothing.
So some of your employees are doing important things. Some are just waiting for the phone to ring. But you don't want them on the couch doing absolutely nothing.