Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I have a process that is reporting in 'top' that it has 6GB of resident memory and 70GB of virtual memory allocated. The strange thing is that this particular server only has 8GB physical and 35GB of swap space available.

From the 'top' manual:

   o: VIRT  --  Virtual Image (kb)
      The total amount of virtual memory used by the  task.   It  includes
      all  code,  data  and  shared  libraries  plus  pages that have been
      swapped out. (Note: you can define the STATSIZE=1 environment  vari-
      able  and  the VIRT will be calculated from the /proc/#/state VmSize
      field.)

      VIRT = SWAP + RES.

Given this explanation, I would expect the virutal memory allocation for a process to be limited to my swap + physical memory available.

According to 'pmap', the code, shared library, and shared memory sections of this process are all minimal - no more than 300M or so.

Obviously, the machine and the process are still functioning correctly (albeit slowly), so what am I missing here?

share|improve this question

4 Answers 4

up vote 7 down vote accepted

It may be demand zero memory which isn't in physical ram, or in the pagefile.

Some resources you may want to look at:

Does your application create a lot of empty memory pages? If so, your application might benefit greatly from:

It allows you to compress and decompress in real-time memory pages. In turn, you are able to keep everything in RAM rather than swap to disk (very slow).

share|improve this answer
    
Yes, the application's doing a lot of correlations in the IPV4 space, so it could potentially have lots of empty pages depending on the traffic distribution. We'll have to watch out for that. Thanks! –  Belly Mar 13 '12 at 20:07
    
glad to help, hopefully some other user will mark me up. I come up with killer answers but I have a rating of 1,266 :-(. I don't think server fault users like me hahhah –  The Unix Janitor Mar 14 '12 at 13:24
    
Few reasons why people may not vote for you: 1. Formatting of your answer---use markup. 2. Your username seems generic. 3. Most importantly: The fact that you find it important enough to comment about it. Leaves a sour taste in people's mouth. –  Belmin Fernandez Mar 14 '12 at 13:39
    
@user37899 Upvotes tend to fall into 3 categories: how informative the answer is, how well-formatted and easy to read it is, and how popular the question is. I'd work on your formatting, but you also have to have some zen and realize that some fantastic answers sit around the site with a mere one upvote -- the popularity of a question is the factor with the most effect. –  Jeff Ferland Mar 14 '12 at 13:41
1  
Did some formatting. Hopefully it helps heh. –  Belmin Fernandez Mar 14 '12 at 13:47

Here's a discussion of virt vs. resident memory:

http://stackoverflow.com/questions/561245/virtual-memory-usage-from-java-under-linux-too-much-memory-used

The discussion refers to Java processes, but is applicable to anything running under Linux. The main point with regard to virt is that the total includes a whole bunch of stuff that may never be used. Virt is something to look at for 32-bit OSes (since processes will hit limits on addressable space), but is largely not useful otherwise. As noted, the thing to pay attention to is the resident memory, which will be limited to available physical RAM and your swap.

share|improve this answer
    
he actually asked why the virtual memory allocated was bigger than his physical memory + swap space.. –  The Unix Janitor Mar 13 '12 at 17:08
    
Yeah, and the discussion over in Stackoverflow talks about how that's possible. –  cjc Mar 13 '12 at 17:09

This is likely because the process' address space is the size as you stated, but it is not really allocated by the OS.

From: http://lwn.net/Articles/428100/

In the process of trying to reach that goal of "low enough overhead and no significant latency," the Go developers have made some simplifying assumptions, one of which is that the memory being managed for a running application comes from a single, virtually-contiguous address range. Such assumptions can run into the same problem your editor hit with vi - other code can allocate pieces in the middle of the range - so the Go developers adopted the same solution: they simply allocate all the memory they think they might need (they figured, reasonably, that 16GB should suffice on a 64-bit system) at startup time.

So that's the unelegant way memory management is done sometimes - having a continous address space simplifies releasing unused mem.

share|improve this answer

The answer is probably MMAP - the data is on the disk, but it is "outside" the swap and can not be seen with "free" or "top" command.

If the java process is not too complicated, you can try play with "lsof" to find where the MMAP file is. However if this java process is complicated, there will be difficult to be seen.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.