4

UPDATE:

This was being caused due to a known issue with nss-softkn. See this article: https://www.splyt.com/blog/2014-05-16-optimizing-aws-nss-softoken


When I run free -m from my centos 6.5 box I am seeing that I only have around 1400 mb of available ram, including the cache. When I do top, sort by memory and add up the processes I am only seeing around 1600 being used. I should have a LOT more free memory. This is happening on a couple of our boxes.

[root@db1 ~]# free -m
              total      used       free     shared    buffers     cached
Mem:          7840       7793         47          0        287       1357
-/+ buffers/cache:       6148       1692
Swap:         7983          7       7976

top output

enter image description here

Here is a better way to visualize the memory consumed by the processes:

enter image description here

9

You have about 1692m in free memory according to Linux.

In the top "Mem" line it would seem like your memory is almost 99% (7793m / 7840m) but in reality you're only using about 78% of available memory. See also http://www.linuxatemyram.com/ for a more in-depth explanation.


Free vs Top vs /proc/meminfo

Process memory.
Adding up the memory used by processes in top and trying to match this with the used memory of free is going to give you a bad time. Due to top paging, you may not even get all the processes in this list a better way to go is to use ps aux --sort -rss to order processes by memory use.

Disk caching.
Linux is going to use the memory you aren't using to make things run much faster for you behind the scenes. This is noted in free's "cached" column and is why you should look at the values next to "-/+ buffers/cache" for actual used and free memory.

Kernel Slab cache.
Kernel modules and drivers have a way to use your unused memory too. The "SReclaimable" is memory that you could use under memory pressure. There's a cool tool called slabtop that can identify how the kernel is this cache.

Clearing caches.

echo 1 > /proc/sys/vm/drop_caches To free pagecache

echo 2 > /proc/sys/vm/drop_caches To free dentries and inodes

echo 3 > /proc/sys/vm/drop_caches To free pagecache, dentries and inodes

See also

  • The slab space was the problem.. I figured that out and added the link below, but since you took the time to detail it in the comment I selected your answer. – ipengineer Dec 17 '14 at 0:36
1

I found the issue here. I needed to look at /proc/meminfo and see what the the kernel was caching as slab space. My problem was not understanding how much "free" memory top/ps/free said I had. It was the fact that these numbers were not adding up. The "free" command would tell me I had roughly 1692mb free but my processes only totaled up to about 20% of my total memory.

This article describes my exact issues:

http://blog.logicmonitor.com/2014/10/09/more-linux-memory-free-memory-that-is-not-free-nor-cache/

0

I believe you are misreading the Buffers/Cache - there is > 6 gigs being used as memory cache as per the +/- Buffers/Cache line. If this is needed it can be eaten into - as its not, the OS is using it to cache disk reads and similar.

  • Is that not saying I have 1692 free after buffers/cache are taken into the equation? 7793-(buffers+cache) = 6148 used. – ipengineer Dec 15 '14 at 23:23
  • Its saying you have used 1692 megs (7840-6148), and 6148 megs is being used as cache. – davidgo Dec 15 '14 at 23:49
0

Based on http://blog.logicmonitor.com/2014/10/09/more-linux-memory-free-memory-that-is-not-free-nor-cache, I created a script that I called free (which I placed in a path ahead of /usr/bin in my .bash_profile) that produces useful results:

#!/usr/bin/perl

$result = `/usr/bin/free`;
print "$result";

if ($result =~ /cache:\s+(\d+)/)
{
 $mem = $1;
 $slab = `cat /proc/meminfo|grep SReclaimable`; $slab =~ s/[^\d]//g;
 print "Slab                  $slab\n";
 print "Actual Used        ** ",$mem-$slab," **\n";
}

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.