Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

My application randomly returned OOM errors trying to allocate 16M chunks, while Linux had plenty memory used by disk cache (20G).

Swapping disabled. All OS limits seem fine.

After clearing Linux cache with drop_caches error disappeared.

Any idea what to check or is it somehow expected behavior?

share|improve this question

closed as off topic by Cakemox, rnxrx, Shane Madden, Tom O'Connor, Michael Hampton Sep 15 '12 at 23:05

Questions on Server Fault are expected to relate to server, networking, or related infrastructure administration within the scope defined by the community. Consider editing the question or leaving comments for improvement if you believe the question can be reworded to fit within the scope. Read more about reopening questions here.If this question can be reworded to fit the rules in the help center, please edit the question.

    
This isn't really a Server Fault question - if you'd like I can move it to Stack Overflow for you. If I had to weigh in though I would do do by slamming a copy of POSIX on the desk and shouting "MALLOC() CAN FAIL. EXPECT IT. DEAL WITH IT IN YOUR CODE!" (It doesn't matter WHY - code that calls malloc() should expect failures & cope with them :) You also shouldn't disable swap on unix systems as a general rule... –  voretaq7 Sep 14 '12 at 13:23
    
1. Stackoverflow will say that it is Linux core question and will move to serverfault? [br] 2. Question is why malloc fails, while Linux has a lot of memory? [br] 3. Swapping makes more problems than "Out of memory message" sometimes. –  nonobe Sep 14 '12 at 13:28
1  
Stack Overflow would be wrong. It would not be the first time the were wrong about migrating things here just because the word "unix" or "server" was present. This is a question about a POSIX call's behavior - that's not system administration, it's C programming. it's tangentially on topic here at best (not bad enough for me to mod-close it, but probably not going to get good answers...) –  voretaq7 Sep 14 '12 at 13:37
    
No, you are wrong. POSIX call returns NULL if OS has not enough memory - but question is why OS thinks it has not enough memory which is hardly related to programming. It is sad if you don't understand it. –  nonobe Sep 18 '12 at 6:26
    
found identical problem, still not answered: serverfault.com/questions/288319/… Identical because Linux calls OOM Killer while a lot of cache is available. In my case Killer is disabled, so just NULL is returned. –  nonobe Sep 18 '12 at 7:22

2 Answers 2

Malloc() does not allocate physical memory, it allocates virtual memory. Malloc() can fail due to lack of (continuous free chunk of) virtual memory or exceeded commit limit.

  1. Check the virtual memory usage of the process using ps, top or pmamp commands. 64bit architectures (amd64) have extremely large virtual memory and is basically impossible to exhaust that, but 32bit process would be limited to at most 4GB of virtual memory.
  2. Check /proc/sys/vm/overcommit_memory and Committed_AS and CommitLimit rows in /proc/meminfo. If overcommit_memory is 1, exceeding CommitLimit will cause malloc() to fail.
share|improve this answer
    
Committed_AS was ~10G lower than CommitLimit. I certainly sure it wasn't any limit because after drop_caches all memory allocations were successful. –  nonobe Sep 14 '12 at 13:10
    
Am I correct that when swapping is disabled, then Virtual memory is almost the same as Physical memory? –  nonobe Sep 14 '12 at 13:26

There was plenty of memory, but probably fragmented so you couldn't get a 16mb continuous chunk. Drop_caches will have triggered a defrag of memory so afterwards there is sufficient continuous memory available to honor your malloc request.

(This question is probably more suited to one of the programming forums.)

share|improve this answer
    
Application requested 16Mb and 20Gb were technically available. So it doesnt look like fragmentation problem. Also it involves Linux memory management, not sure if that is programming question –  nonobe Sep 14 '12 at 11:15

Not the answer you're looking for? Browse other questions tagged or ask your own question.