The Stack Overflow podcast is back! Listen to an interview with our new CEO.
56

vmtouch seems like a good tool for the job. Highlights: query how much of a directory is cached query how much of a file is cached (also which pages, graphical representation) load file into cache remove file from cache lock files in cache run as daemon vmtouch manual EDIT: Usage as asked in the question is listed in example 5 on vmtouch Hompage Example ...


26

This is also possible using the vmtouch Virtual Memory Toucher utility. The tool allows you to control the filesystem cache on a Linux system. You can force or lock a specific file or directory in the VM cache subsystem, or use it to check to see what portions of a file/directory are contained within VM. How much of the /bin/ directory is currently in ...


18

kswapd is managing swap space in response to memory demands greater than physically available for all processes. It is process agnostic, it is only interested in what pages are access and when (it is more complex than this of course but to keep things simple we may as well view it this way). So the real question is "what processes have the greatest burden ...


10

I don't understand this question - you've disabled the page file (Which, to my knowledge, isn't recommended practice even in a virtual environment) and now you're asking how you can provide a page file? Why don't you just re-enable the page file on this one server, and set the size manually?


9

More (moar) RAM is always better, if the path to obtaining it is clear. In most VMWare deployments, you'll run out of RAM far sooner than your CPU resources. I'd suggest providing the lowest reasonable amount of RAM for each specific VM, making sure the VMWare tools are installed and running within each VM. This controls the balloon driver. One way of ...


9

Just for a sense of completeness I'll drop this in as an answer for future reference/searches. Basically look at the VM's settings, then choose Resources, click on Memory and be sure that the 'Unlimited' box is ticked - leaving this unticked can cause needless ballooning.


9

It may be demand zero memory which isn't in physical ram, or in the pagefile. Some resources you may want to look at: http://www.gitam.edu/eresource/comp/gvr(os)/13.6.htm http://linux.derkeiler.com/Newsgroups/comp.os.linux.development.system/2006-11/msg00176.html Does your application create a lot of empty memory pages? If so, your application might ...


9

You can script it.. but you can also do it via top Run top then press O followed by p then enter Now all the processes are sorted by swap usage and you can see which ones are using it


8

If you're on Ubuntu 15.10 or greater, this may actually be the result of a bug, especially if your system is a virtual machine lacking a swap partition (e.g., AWS EC2). The problem exists on other distributions, but, as of writing, it's unclear if the same fix works universally. A temporary workaround: sudo ln -s /dev/null /etc/udev/rules.d/40-vm-hotadd....


8

The problem is explained here: Jun 19 21:29:49 server-name kernel: [17009377.877956] bash invoked oom-killer: gfp_mask=0x26000c0, order=2, oom_score_adj=0 And here: Jun 19 21:29:49 server-name kernel: [17009377.878115] Node 1 Normal: 50883*4kB (UME) 133*8kB (UM) 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 204596kB The kernel ...


7

You get a list of things you can sort by by pressing O. Virtual segment size is option o, so you can get what you want by pressing Oo<Enter>.


7

There are two kernel settings that can help considerably even without using other tools: swappiness tells linux kernel how aggressively it should use swap. Quoting the Wikipedia article: Swappiness is a property for the Linux kernel that changes the balance between swapping out runtime memory, as opposed to dropping pages from the system page cache. ...


7

For anyone needing an even more simple, step-by-step explanation, check http://thoughtsbyclayg.blogspot.com/2008/09/display-free-memory-on-linux-ubuntu.html, his explanation helped me a lot.


6

There also seems to be a bug in kswapd somewhere, hopefully only on older kernels. Nearly each day now kswapd goes beserk randomly on some machines in a bigger cluster (with a non-current kernel, though). 100% CPU on both kswapd processes. No other running processes (except ssh shell), plenty of free RAM (more than 700 MB) and no SWAP used at all. No ...


6

This is actually a known feature of Windows XP/2003. I'm not sure if Windows 7/2008/R2 retained this behavior. The working set of an application is trimmed when its top-level window is minimized http://support.microsoft.com/kb/293215


6

Try pressing F or O in top, that should allow you to select the column you want to sort by.


6

Im coming back here to answer my own question! The issue is caused when you have multi cpu's (not multi core) eg 2 cpus each with x cores vm.zone_reclaim_mode=0 fixes this.


5

You can get that sort of information from the /proc/<pid>/smaps file for each process in the form of the Pss entry (short for "Proportional share size"). In the above example with 200MB "shared" between two processes, each process would show 100MB in the PSS entry for that mapping, i.e. the memory is distributed evenly between the processes that share ...


5

I was looking for more clear description about buffer and i found in "Professional Linux® Kernel Architecture 2008" Chapter 16: Page and Buffer Cache Interaction Setting up a link between pages and buffers serves little purpose if there are no benefits for other parts of the kernel. As already noted, some transfer operations to and from ...


5

Not necessarily a problem. That virtual memory profile is what we see for the same architecture and physical memory usage. If it were to escalate indefinitely, that would be bad and it would eventually terminate, although the operating system limit is quite high. If a large virtual memory allocation were performed, and the system did not have a large ...


4

I think AndreasM has hit it on the head (the reason for the disk going all thrashy.) Executables are demand paged -- so in normal operation you will have nearly all of your executables nad libraries sitting in good ol' physical RAM. But when RAM runs low, but not low enough for the out-of-memory killer to be run, these pages are evicted from RAM. So you ...


4

(everything ewwhite said) plus: Memory overcommit will probably not help you at all, as you're going to run a mix of different sets of data in RAM (different operating systems). It only helps when you have lots and lots of nearly identical stuff in data (like a big VDI server where everyone has the exact same master image).


4

MySQL has only mapped the memory pages, and is not actually using them. VIRT != memory use. Linux memory usage can be very confusing sometimes. You can see from your top output by yourself that no swap has been used, and RAM is mostly unused too.


4

The VIRT reflects to amount of Virtual Memory allocated by a process, not a real one. The address space of a process may contain certain areas mapped to corresponding files which are not located in RAM. At least the program executable and shared libraries are mapped to the virtual address space of a process. During the program execution the process may map ...


4

OpenVZ providers typically sell RAM capacity of their virtual machines as two numbers: "guaranteed" and "burst" RAM. You are supposed to always get the "guaranteed" amount of RAM, and may use up to the "burst" amount if host resources permit. For example a VPS might be sold as "512MiB guaranteed 1GiB burst RAM". You should always have a successful memory ...


4

This is just a guess, but it's too long for a comment, so here goes. With virtual private servers, you get a "slice" of a real server. Rather than spending, say, $500 each on 24 small servers with 512MB ram etc, it's cheaper to buy (and operate) one $3500 server with 12GB RAM and two six-core hyperthreaded cpus. Here's the trick: that server might sell ...


4

Total virtual memory = physical memory + pagefile size "Virtual memory" refers to the total amount of memory that programs can potentially use on the machine. The pagefile serves as an extension of physical memory, and that's where unused code and data is swapped to in order to free up physical memory for other programs that may need it. There is less "...


4

Nonpaged pool is kernel memory which can't be paged out into the pagefile when Windows runs out of free physical memory. It is used by drivers to allocate memory which they need. Post some pictures of taskmgr, when you into the issue. If the usage is too large, use poolmon to find the causing driver of the leak.


3

Your message is generated by the slab allocator, the mechanism used by the Linux kernel to manage the portion of memory it needs for itself (not for user space processes). Originally it was written by Jeff Bonwick for Solaris, and it's probably easiest to understand by reading Bonwick(94) (PDF). If you are wondering about the name, in his blog Bonwick ...


3

Explained by RedHat: Cache Pages: A cache is the part of the memory which transparently stores data so that future requests for that data can be served faster. This memory is utilized by the kernel to cache disk data and improve i/o performance. The Linux kernel is built in such a way that it will use as much RAM as it can to cache information from your ...


Only top voted, non community-wiki answers of a minimum length are eligible