New 64 bit linux system has regular processes (ps, grep etc) taking up way too much VIRT mem

Question

We just moved from a 32-bit machine to a 64-bit machine. We have quickly ran out of memory despite the new boxes have twice as much ram as the old boxes.

Running a simple ps command will illustrate the problem.

New machine:

132 prod-Charlotte1-node1 ~/public_html/rearch/cgi-bin> ps aux | grep ps
root       293  0.0  0.0      0     0 ?        S<   May09   0:00 [kpsmoused]
xamine    2267  1.0  0.0  63728   928 pts/3    R+   16:50   0:00 ps aux
xamine    2268  0.0  0.0  61172   752 pts/3    S+   16:50   0:00 grep ps

Old machine:

132 prod-116431-node1:/home/xamine> ps aux | grep ps
xamine   23191  0.0  0.0  2332  768 pts/6    R+   15:41   0:00 ps aux
xamine   23192  0.0  0.0  3668  692 pts/6    S+   15:41   0:00 grep ps

Notice that the ps process is using 63M of VIRT mem vs 2 on the old machine.

New Machine:

• Enterprise Linux Enterprise Linux Server release 5.4 (Carthage)
• Red Hat Enterprise Linux Server release 5.4 (Tikanga)

Old Machine:

• Red Hat Enterprise Linux ES release 4 (Nahant Update 4)

Answer 1

Depends on how you are counting used memory. If you are looking at "free", make sure to discount the cached and buffers used.

Linux tries to cache as much disk activity as it can so that subsequent accesses to those files is much faster than having to go to the disk again. If memory is required, cache memory will be released to satisfy the new request.

Eg:

# free
         total       used       free     shared    buffers     cached
Mem:       3973040    3944864      28176          0     433448    3123468
-/+ buffers/cache:     387948    3585092
Swap:      2040244      72080    1968164

In this case, while the system is reporting almost all 4G of memory as used, closer examination shows that 3G of it is "cached", meaning that there is actually plenty of memory available. The second line of the free output represents that calculation -- excluding buffers and cache, there is 3.5G available memory.

Answer 2

The virt memory number is misleading. It includes the size of all the shared libraries that the program links against. These libraries, by virtue of being shared, are loaded only once in to system memory for all the programs that use them.

A better measure of memory usage for your process in this case is the Resident Set Size (RSS) which is the column after the virtual memory. This is the amount of physical memory your application is using. Assuming you are not going in to swap, which for a program like ps is not likely, this is a good measure of how much "actual" memory the application is using in this case. By that metric, the difference is largely negligible.

The reason for the large difference in virtual size could be for any number of reasons. Part of it is likely due to the larger size of types, especially pointers, in a 64 versus 32 bit system. The other reason may simply be because of an increase in the size of the libraries, or perhaps linking to a different number of libraries.

Maybe if you gave a more representative sample of what is actually running on these machines, it would be more helpful in pinpointing why you are out of memory.