We just moved from a 32-bit machine to a 64-bit machine. We have quickly ran out of memory despite the new boxes have twice as much ram as the old boxes.

Running a simple ps command will illustrate the problem.

New machine:

132 prod-Charlotte1-node1 ~/public_html/rearch/cgi-bin> ps aux | grep ps
root       293  0.0  0.0      0     0 ?        S<   May09   0:00 [kpsmoused]
xamine    2267  1.0  0.0  63728   928 pts/3    R+   16:50   0:00 ps aux
xamine    2268  0.0  0.0  61172   752 pts/3    S+   16:50   0:00 grep ps

Old machine:

132 prod-116431-node1:/home/xamine> ps aux | grep ps
xamine   23191  0.0  0.0  2332  768 pts/6    R+   15:41   0:00 ps aux
xamine   23192  0.0  0.0  3668  692 pts/6    S+   15:41   0:00 grep ps

Notice that the ps process is using 63M of VIRT mem vs 2 on the old machine.

New Machine:

  • Enterprise Linux Enterprise Linux Server release 5.4 (Carthage)
  • Red Hat Enterprise Linux Server release 5.4 (Tikanga)

Old Machine:

  • Red Hat Enterprise Linux ES release 4 (Nahant Update 4)

Depends on how you are counting used memory. If you are looking at "free", make sure to discount the cached and buffers used.

Linux tries to cache as much disk activity as it can so that subsequent accesses to those files is much faster than having to go to the disk again. If memory is required, cache memory will be released to satisfy the new request.


# free
         total       used       free     shared    buffers     cached
Mem:       3973040    3944864      28176          0     433448    3123468
-/+ buffers/cache:     387948    3585092
Swap:      2040244      72080    1968164

In this case, while the system is reporting almost all 4G of memory as used, closer examination shows that 3G of it is "cached", meaning that there is actually plenty of memory available. The second line of the free output represents that calculation -- excluding buffers and cache, there is 3.5G available memory.

  • Doesn't look like that is the case here:<br /> 132 prod-Charlotte1-node1 ~/public_html/rearch/cgi-bin> free -m<br /> total used free shared buffers cached<br /> Mem: 16000 15856 143 0 31 53<br /> -/+ buffers/cache: 15771 228<br /> Swap: 19455 1308 18147<br /> – user42980 May 13 '10 at 15:00

The virt memory number is misleading. It includes the size of all the shared libraries that the program links against. These libraries, by virtue of being shared, are loaded only once in to system memory for all the programs that use them.

A better measure of memory usage for your process in this case is the Resident Set Size (RSS) which is the column after the virtual memory. This is the amount of physical memory your application is using. Assuming you are not going in to swap, which for a program like ps is not likely, this is a good measure of how much "actual" memory the application is using in this case. By that metric, the difference is largely negligible.

The reason for the large difference in virtual size could be for any number of reasons. Part of it is likely due to the larger size of types, especially pointers, in a 64 versus 32 bit system. The other reason may simply be because of an increase in the size of the libraries, or perhaps linking to a different number of libraries.

Maybe if you gave a more representative sample of what is actually running on these machines, it would be more helpful in pinpointing why you are out of memory.

  • We have a perl process running that on the old system uses about 50-60 MB VIRT and a little less RSS. On the new machines, the now take up about 180MB VIRT and 60-70 RSS. This is causing the machine to run out of memory and go to the swap. 132 prod-Charlotte1-node1 ~/public_html/rearch/cgi-bin> free -m total used free shared buffers cached Mem: 16000 15856 143 0 31 53 -/+ buffers/cache: 15771 228 Swap: 19455 1308 18147 – user42980 May 13 '10 at 15:05
  • How is one Perl process using 180MB of virtual memory causing a machine with 16GB of RAM to run out of memory? You're still missing something here. – Kamil Kisiel May 13 '10 at 16:27
  • Well, there are 200 of them running :) – user42980 May 13 '10 at 20:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.