61

You are experiencing the Linux Ate My Ram issue. Don't Panic. This is NOT a problem. Your system is Working As Designed. The problem is not your OS -- the problem is your understanding of what "free" memory is. Unix systems use memory for more than just running programs. Memory might be used for: Running programs (active/used) Buffering data in ...


31

Setting overcommit_ratio to 80 is likely not the right action. Setting the value to anything less than 100 is almost always incorrect. The reason for this is that linux applications allocate more than they really need. Say they allocate 8kb to store a couple character string of text. Well thats several KB unused right there. Applications do this a lot, and ...


27

This is the same "problem" as from Server refuses to use swap partition and a few other similar questions on this site. ( High Memory Usage on Linux Server, Memory Usage in LINUX, Web Server Running Low in Memory, etc.) Pay attention to the fact that the memory consumption is from cache. This means it's keeping a file in memory. Cached memory is "free" ...


24

Well, there is an easy way to take a look at the kernel's page cache if you happen to have ftools - "fincore" gives you some summary information on what files' pages are the content of the cache. You will need to supply a list of file names to check for their presence in the page cache. This is because the information stored in the kernel's page cache ...


23

Section 9.6 "Overcommit and OOM" in the doc that @dunxd mentions is particularly graphic on the dangers of allowing overcommit. However, the 80 looked interesting to me as well, so I conducted a few tests. What I found is that the overcommit_ratio affects the total RAM available to ALL processes. Root processes don't seem to be treated differently from ...


23

OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME 146851887 146851887 12% 0.19K 6992947 21 27971788K dentry You say it's not because of disk caching, but clearly it is. My bet is that you have code that makes lots of fetches for files that do not exist and you get a ton of negative caching. Linux will remove these entries if it's under ...


23

Generally, I'm opposed to the idea that a Windows server should be rebooted on a regular schedule EXCEPT in relation to TS/RDS servers. We reboot ours every day. It clears up old sessions, releases in use resources (CPU, RAM, file handles, etc.), so my opinion and suggestion would be that you do configure a daily scheduled reboot of your RDS servers. Note ...


17

You can use the vmtouch utility to see if a named file or directory is in cache. You can also use the tool to force items into cache or lock them into cache. [root@xt ~]# vmtouch -v /usr/local/var/orca/procallator.cfg /usr/local/var/orca/procallator.cfg [ ] 0/5 Files: 1 Directories: 0 Resident Pages: 0/5 0/20K 0% Elapsed: 0....


17

You can use the zpool status -D poolname command. The output would look similar to: root@san1:/volumes# zpool status -D vol1 pool: vol1 state: ONLINE scan: scrub repaired 0 in 4h38m with 0 errors on Sun Mar 24 13:16:12 2013 DDT entries 2459286, size 481 on disk, 392 in core bucket allocated referenced ...


17

Users refuse to log off instead of disconnect Setup the appropriate group policies to auto-logoff them. You can separately control an idle timeout and logoff. That should certainly minimize some of the issue during the day. I restart my 3 server TS farm daily at 3:00am. Because, yes crap can build up over time when you have lots of people using a single ...


14

This is what I use for an approximation of the average httpd (substitute apache2 if on Debian distro) process size: ps -ylC httpd --sort:rss | awk '{sum+=$8; ++n} END {print "Tot="sum"("n")";print "Avg="sum"/"n"="sum/n/1024"MB"}' Like symcbean said you should take about 80% of the server's memory and divide it by the average process size to determine your ...


13

I had the same experience in Nagios when I used the check_mem.pl plugin. When you define the tress hold for critical, you need to use a capital C instead of a normal c. Like this: command[check_mem]=/usr/lib/nagios/plugins/check_mem -w 10 -C 5 -f Then it takes the cached memory in account and it will not send warnings.


12

An easy way to see if you need more RAM is to chart the Page Life Expectancy perfmon counter. This counter tells you how long SQL Server thinks that data will be kept in the buffer pool before it needs to make room for other data. You want this number as high as possible. With 6 Gigs of RAM installed (you should have SQL set to max out at probably 4 gigs) ...


12

ZFS is likely using most of your memory as ARC cache. Should you want to know how your RAM is used, run this command as root: # echo ::memstat | mdb -k On Solaris 10 10/09 and newer, this displays something like this: Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel ...


11

Your smem -tw output shows that your kernel is consuming over 9 GB of dynamic memory: danslimmon@bad-server:~$ smem -tw Area Used Cache Noncache firmware/hardware 0 0 0 kernel image 0 0 0 kernel dynamic memory 12857576 2887440 9970136 ...


9

Solved with the following: cat <<EOF > /etc/profile.d/ImageMagick.sh # Set ImageMagick memory limits: it eats too much export MAGICK_MEMORY_LIMIT=1024 # Use up to *MB of memory before doing mmap export MAGICK_MAP_LIMIT=1024 # Use up to *MB mmaps before caching to disk export MAGICK_AREA_LIMIT=4096 # Use up to *MB disk space before failure ...


9

It's high because that saves effort. It takes effort to make memory free. And if you do that, it just takes effort to make it used again. So, to save effort, modern operating systems only make memory free if they have absolutely no other choice. If you're thinking "I want memory free now so I can use it later", banish that thought from your mind. Memory ...


9

You have about 1692m in free memory according to Linux. In the top "Mem" line it would seem like your memory is almost 99% (7793m / 7840m) but in reality you're only using about 78% of available memory. See also http://www.linuxatemyram.com/ for a more in-depth explanation. Free vs Top vs /proc/meminfo Process memory. Adding up the memory used by ...


8

I have discovered the answer to my own question - thanks to womble's help (submit an answer if you like). lsof -s shows file handles in use, and turns out there were several gigabytes of mmap'd log files taking up the cache. Implementing a logrotate should resolve the issue completely and allow me to take advantage of more memory. I will also re-enable ...


8

You have earned Confused by Linux memory usage badge! Processes are not the only thing eating up memory in Linux. The frequently used files and directory entries are cached in RAM - that's because reading the files from RAM is lots faster than reading them from disk. I bet you didn't take buffers and cached columns into account while calculating the RAM ...


8

After re-reading your comments above and your original post - you ran ulimit -a as root - but you said that root is unaffected. Your issue here is ulimit, but at a user level. Log in as your dan user and run ulimit -a and you will probably find your memory is limited.


8

As HTTP500 said, you probably shouldn't worry about it unless the machine is actually choking. Take a look at Help! Linux Ate My RAM!. It's not quite an analogous situation, as you have processes rather than buffers using up the RAM, but it's not too far off: your machine's physical RAM is a resource to be used, either with kernel buffers or spare Apache ...


8

Memory management is already a complex topic, and unfortunately, Microsoft has altered its exact definitions and terms regarding different types of memory with pretty much every single release of Windows. So trust me, you aren't the only one confused by it. When you use tasklist, tasklist shows you the Working Set that is assigned to each process. A ...


8

Memory fragmentation is a non-trivial issue. Before v4, the only way to resolve it was restarting the process (possibly after making a slave, promoting it and redirecting traffic to it). As of v4, there's an experimental active memory defragmentation mechanism that may be enabled with a simple CONFIG SET activedefrag yes.


7

For anyone needing an even more simple, step-by-step explanation, check http://thoughtsbyclayg.blogspot.com/2008/09/display-free-memory-on-linux-ubuntu.html, his explanation helped me a lot.


7

The short answer is that the rest of your memory is used to cache your applications. The long answer can be found here.


7

This is perfectly normal. If you want memory not to be used, take it out of the server and put it on your desk. Free memory is wasted memory. Linux, like most modern operating systems, can directly transition memory from one use to another. Making it free in the middle has costs associated with it and no benefits associated with it. So sensible operating ...


7

You really don't want to do this. The kernel will immediately relinquish cache pages to other applications if and when they call for it. It's a complete waste of money to not let the kernel use all your available RAM.


7

The -r flag for sar will give you kbmfree, kbmemused, kbbuffers, and kbcached. It doesn't, unfortunately, total them like the -/+ buffers/cache column like the free command, so you will have to pipe it through an awk or perl script to total them. Try this: sar -r |\ awk ' { if (NR > 3 && $3 != "kbmemfree" && $3 != "") { if ($...


7

I'd never heard of it before your mention of it. In an article on SCO Unix, My system is slow (Old Sco Unix) on the utility, though, it doesn't sound like it is apt for what you want to do. Linux systems have "memhog" if numactl is installed. Numactl can assign specific a process to a specific cpu and control memory policy in other ways. The "memhog" ...


Only top voted, non community-wiki answers of a minimum length are eligible