Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

We are running a heavy service on Debian. Our service generates lots of file i/o and its performance relies on how well Linux kernel caches files.

Our system has 64GB memory. Total file size is 500GB (lots of small files). These files are served via reiserfs on raid. The kernel is 3.2.9 now. (this is not a new issue. We have this issue for several years.)

Usually, most of file content are cached and our system runs smoothly. free(1) command shows 14G memory cache, 10G memory buffer, and 8G memory are free.

However, in our peak hour, I found linux kernel released most of cache/buffer and kept them in free state. In my opinion, it is a huge waste to keeps memory in free state and I guess it is the reason that our service overloads in busy hours. free(1) command show something like 2G cache, 2G buffer, and 28G free.

My question is, in what situation linux kernel want to release memory instead of caching/buffering. And how to avoid this behavior? Is there any configuration we can tune?

(Let me know if you need more information about our hardware/system/service)

share|improve this question
3  
Memory used by system file caches will be made available to other processes that need it. So, under high load, it is possible that your server's processes need more memory, and so reduce the memory used by the caches/buffers. Once load drops, you now have 'extra' memory free, until your caches/buffer fill up again. A graph of memory usage (broken down by free/used/buffers/caches) over time will show you if this is true or not (i.e. an increase in used memory proceeding the decrease in cache/buffers). (Post the graph if you have one). –  cyberx86 Apr 21 '12 at 18:08
    
"I guess it is the reason that our service overloads in busy hours." — you quite possible could have wrong assumption. –  poige Apr 21 '12 at 22:20

2 Answers 2

Is it possible to put anything in a ramdisk? Explicit cache if you will. Not sure if what you're doing suits this, and you have probably thought aboout it, but i'll throw it out there ;)

Have a look here: http://www.linuxinsight.com/proc_sys_vm_hierarchy.html

there are a few bits you can tune easily, just by "cat"ing values in. vfs_cache_pressure & zone_reclaim_mode jumps out, caveat: only one i have ever tuned is swappiness.

share|improve this answer

There are two keys reasons this would happen.

One, that I hope you can rule out, is some person or script commanding it to do so. For example, someone sending a command to /sys/vm/drop_caches.

The other is if that memory is required for some other purpose. If an application requires a lot of memory, the kernel will give the application that memory, reducing the cache as needed. When the application is done with that memory, it will become free. It will only fill back up with cached data over time.

There may be a few things you can do if this is the case. For example, if the memory usage is happening because you're trying to process four requests at once, you may wish to reconfigure to do two requests at once maximum. If this lets you keep more data in cache, it may be a performance win overall.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.