I have a qnap nas unit that I was trying to create an additional iscsi target on, and the creation process fails. Im not trying to do anything I haven't done on other identical units, but this one has been running for longer.
ssh in and have a look
dmesg contains amongst many other things:
[3431220.016334] SLAB: Unable to allocate memory on node 0 (gfp=0xd0)
[3431220.016335] cache: kmalloc-65536, object size: 65536, order: 4
[3431220.016341] node 0: slabs: 12/12, objs: 12/12, free: 0
Ok we got the cause of the problem.
# cat /proc/meminfo
MemTotal: 3939760 kB
MemFree: 190480 kB
Buffers: 3212476 kB
Cached: 188868 kB
SwapCached: 48 kB
So all memory allocated to buffers. Hardly surprising on a linux machine that does little other than block IO
# cat /proc/buddyinfo
Node 0, zone DMA 8 2 1 1 1 1 1 1 2 2 2
Node 0, zone DMA32 22614 389 0 1 1 2 3 1 0 0 0
Node 0, zone Normal 12150 7683 0 2 5 2 1 1 0 0 0
ouch. This is showing some severe memory fragmentation, with virtually no blocks greater than 8K available. Is there any way of clearing the memory fragmentation problem (temporarily reducing the memory allocated to buffers or otherwise) short of restarting the machine, a process that is going to be hugely inconvenient? Also, is it likely that adding more memory to a machine used like this is going to help this particular problem, there are obviously other advantages to upgrading the memory.