Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
61

You are experiencing the Linux Ate My Ram issue. Don't Panic. This is NOT a problem. Your system is Working As Designed. The problem is not your OS -- the problem is your understanding of what "free" memory is. Unix systems use memory for more than just running programs. Memory might be used for: Running programs (active/used) Buffering data in ...


31

Setting overcommit_ratio to 80 is likely not the right action. Setting the value to anything less than 100 is almost always incorrect. The reason for this is that linux applications allocate more than they really need. Say they allocate 8kb to store a couple character string of text. Well thats several KB unused right there. Applications do this a lot, and ...


27

This is the same "problem" as from Server refuses to use swap partition and a few other similar questions on this site. ( High Memory Usage on Linux Server, Memory Usage in LINUX, Web Server Running Low in Memory, etc.) Pay attention to the fact that the memory consumption is from cache. This means it's keeping a file in memory. Cached memory is "free" ...


23

Well, there is an easy way to take a look at the kernel's page cache if you happen to have ftools - "fincore" gives you some summary information on what files' pages are the content of the cache. You will need to supply a list of file names to check for their presence in the page cache. This is because the information stored in the kernel's page cache ...


23

Section 9.6 "Overcommit and OOM" in the doc that @dunxd mentions is particularly graphic on the dangers of allowing overcommit. However, the 80 looked interesting to me as well, so I conducted a few tests. What I found is that the overcommit_ratio affects the total RAM available to ALL processes. Root processes don't seem to be treated differently from ...


23

OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME 146851887 146851887 12% 0.19K 6992947 21 27971788K dentry You say it's not because of disk caching, but clearly it is. My bet is that you have code that makes lots of fetches for files that do not exist and you get a ton of negative caching. Linux will remove these entries if it's under ...


23

Generally, I'm opposed to the idea that a Windows server should be rebooted on a regular schedule EXCEPT in relation to TS/RDS servers. We reboot ours every day. It clears up old sessions, releases in use resources (CPU, RAM, file handles, etc.), so my opinion and suggestion would be that you do configure a daily scheduled reboot of your RDS servers. Note ...


17

You can use the zpool status -D poolname command. The output would look similar to: root@san1:/volumes# zpool status -D vol1 pool: vol1 state: ONLINE scan: scrub repaired 0 in 4h38m with 0 errors on Sun Mar 24 13:16:12 2013 DDT entries 2459286, size 481 on disk, 392 in core bucket allocated referenced ...


17

Users refuse to log off instead of disconnect Setup the appropriate group policies to auto-logoff them. You can separately control an idle timeout and logoff. That should certainly minimize some of the issue during the day. I restart my 3 server TS farm daily at 3:00am. Because, yes crap can build up over time when you have lots of people using a single ...


16

You can use the vmtouch utility to see if a named file or directory is in cache. You can also use the tool to force items into cache or lock them into cache. [root@xt ~]# vmtouch -v /usr/local/var/orca/procallator.cfg /usr/local/var/orca/procallator.cfg [ ] 0/5 Files: 1 Directories: 0 Resident Pages: 0/5 0/20K 0% Elapsed: 0....


15

Running top in batch mode to report memory sizes periodically can be used to see who is using the memory when things go south. Runing sar in batch mode should give some good diagnostics on memory use, and related I/O. Running munin to monitor the system should give you a graph with good detail on what memory is being used for. This may help a lot. You ...


14

This is what I use for an approximation of the average httpd (substitute apache2 if on Debian distro) process size: ps -ylC httpd --sort:rss | awk '{sum+=$8; ++n} END {print "Tot="sum"("n")";print "Avg="sum"/"n"="sum/n/1024"MB"}' Like symcbean said you should take about 80% of the server's memory and divide it by the average process size to determine your ...


13

I had the same experience in Nagios when I used the check_mem.pl plugin. When you define the tress hold for critical, you need to use a capital C instead of a normal c. Like this: command[check_mem]=/usr/lib/nagios/plugins/check_mem -w 10 -C 5 -f Then it takes the cached memory in account and it will not send warnings.


12

An easy way to see if you need more RAM is to chart the Page Life Expectancy perfmon counter. This counter tells you how long SQL Server thinks that data will be kept in the buffer pool before it needs to make room for other data. You want this number as high as possible. With 6 Gigs of RAM installed (you should have SQL set to max out at probably 4 gigs) ...


12

The main parameter for tweaking Apache's memory usage will be MaxClients. Too low a value and you will run out of available slots to serve client requests. Too many and you will use up all your RAM and begin to use swap space which will kill performance (it may appear to be a server crash). One way of tuning MaxClients is to observe the system's memory ...


12

ZFS is likely using most of your memory as ARC cache. Should you want to know how your RAM is used, run this command as root: # echo ::memstat | mdb -k On Solaris 10 10/09 and newer, this displays something like this: Page Summary Pages MB %Tot ------------ ---------------- ---------------- ---- Kernel ...


11

You can use this command to see the top 10 applications regarding RAM usage: ps -A --sort -rss -o comm,pmem | head -n 11 Sometimes this command helps you if many sub processes have been generated: ps auxf This way you can see which processes belong together.


11

Your smem -tw output shows that your kernel is consuming over 9 GB of dynamic memory: danslimmon@bad-server:~$ smem -tw Area Used Cache Noncache firmware/hardware 0 0 0 kernel image 0 0 0 kernel dynamic memory 12857576 2887440 9970136 ...


9

Solved with the following: cat <<EOF > /etc/profile.d/ImageMagick.sh # Set ImageMagick memory limits: it eats too much export MAGICK_MEMORY_LIMIT=1024 # Use up to *MB of memory before doing mmap export MAGICK_MAP_LIMIT=1024 # Use up to *MB mmaps before caching to disk export MAGICK_AREA_LIMIT=4096 # Use up to *MB disk space before failure ...


9

It's high because that saves effort. It takes effort to make memory free. And if you do that, it just takes effort to make it used again. So, to save effort, modern operating systems only make memory free if they have absolutely no other choice. If you're thinking "I want memory free now so I can use it later", banish that thought from your mind. Memory ...


9

You have about 1692m in free memory according to Linux. In the top "Mem" line it would seem like your memory is almost 99% (7793m / 7840m) but in reality you're only using about 78% of available memory. See also http://www.linuxatemyram.com/ for a more in-depth explanation. Free vs Top vs /proc/meminfo Process memory. Adding up the memory used by ...


8

Nothing is really using that memory in terms of applications. You need to deduct the 'cached' value which represents the page cache to get a better idea as to what your actual memory usage is in terms of program usage. Basically this is good memory management and this is ideally what you want. See the link here for more infomation: http://www....


8

I have discovered the answer to my own question - thanks to womble's help (submit an answer if you like). lsof -s shows file handles in use, and turns out there were several gigabytes of mmap'd log files taking up the cache. Implementing a logrotate should resolve the issue completely and allow me to take advantage of more memory. I will also re-enable ...


8

You have earned Confused by Linux memory usage badge! Processes are not the only thing eating up memory in Linux. The frequently used files and directory entries are cached in RAM - that's because reading the files from RAM is lots faster than reading them from disk. I bet you didn't take buffers and cached columns into account while calculating the RAM ...


8

After re-reading your comments above and your original post - you ran ulimit -a as root - but you said that root is unaffected. Your issue here is ulimit, but at a user level. Log in as your dan user and run ulimit -a and you will probably find your memory is limited.


8

As HTTP500 said, you probably shouldn't worry about it unless the machine is actually choking. Take a look at Help! Linux Ate My RAM!. It's not quite an analogous situation, as you have processes rather than buffers using up the RAM, but it's not too far off: your machine's physical RAM is a resource to be used, either with kernel buffers or spare Apache ...


8

Memory management is already a complex topic, and unfortunately, Microsoft has altered its exact definitions and terms regarding different types of memory with pretty much every single release of Windows. So trust me, you aren't the only one confused by it. When you use tasklist, tasklist shows you the Working Set that is assigned to each process. A ...


8

Memory fragmentation is a non-trivial issue. Before v4, the only way to resolve it was restarting the process (possibly after making a slave, promoting it and redirecting traffic to it). As of v4, there's an experimental active memory defragmentation mechanism that may be enabled with a simple CONFIG SET activedefrag yes.


7

For anyone needing an even more simple, step-by-step explanation, check http://thoughtsbyclayg.blogspot.com/2008/09/display-free-memory-on-linux-ubuntu.html, his explanation helped me a lot.


7

The short answer is that the rest of your memory is used to cache your applications. The long answer can be found here.


Only top voted, non community-wiki answers of a minimum length are eligible