Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

In our servers we have a habit of dropping caches at midnight.

sync; echo 3 > /proc/sys/vm/drop_caches

When I run the code it seems to free up lots of RAM, but do I really need to do that. Isn't free RAM a waste?

share|improve this question
45  
Find the person who put this in and ask him why he did it. As you correctly guessed, there is no obvious good reason for it. –  Michael Hampton May 20 at 3:15
7  
Debugging the kernel. That's about it. This doesn't actually free up any RAM; it drops caches, as the name suggests, and thus reduces performance. –  Michael Hampton May 20 at 3:38
22  
@ivcode Then you should find and fix the problem with that server rather than trying to avoid the conditions that cause it. If my car stalled every time I made a sharp right turn, avoiding sharp right turns is a lousy fix. –  David Schwartz May 20 at 5:01
5  
Related thedailywtf.com/Articles/Modern-Memory-Management.aspx Strongly arguing it's a bad idea. –  Drunix May 20 at 9:22
6  
Related, and a useful description of the "problem": linuxatemyram.com –  Bill Weiss May 20 at 16:47

10 Answers 10

You are 100% correct. It is not a good practice to free up RAM. This is likely an example of cargo cult system administration.

share|improve this answer
6  
+1 for mentioning Cargo Cult System Administration. Any sysadmin who doesn't know that term and what it means should be fired. –  Tonny May 20 at 10:22
7  
@Tonny: We would be left without sysadmin department then :( –  PlasmaHH May 20 at 19:44
    
@PlasmaHH Unfortunately that is the situation most us find themselves in... The one thing worse than a sysadmin who doesn't know about CargoCult is an ICT manager who doesn't know AND is who runs his department like CargoCult. I worked for such a one once upon a time. I left that place after 3 months and I swore NEVER to do that again. (They went bankrupt 7 months later... Partly due to ICT mismanagement breaking their Sales system beyond repair.) –  Tonny May 20 at 19:56
4  
Surely it's at least plausible that one could fully see and agree such practices are bad without knowing that specific disparaging term for them? I guess arguably, any sysadmin who doesn't spend time on the kind of forums where people are continually criticising cargo-cult practices as such, isn't keeping their knowledge current ;-) –  Steve Jessop May 23 at 10:55
    
Like most of humanity, I love terse brash assertions with lots of approval, but a cite or reasoning would earn my superego's +1. –  Aaron Hall May 23 at 20:22

Yes, clearing cache will free RAM, but it causes the kernel to look for files on the disk rather than in the cache which can cause performance issues.

Normally the kernel will clear the cache when the available RAM is depleted. It frequently writes dirtied content to disk using pdflush.

share|improve this answer
13  
+1 for explaining why it's a bad idea. –  Ogre Psalm33 May 20 at 17:00

The reason to drop caches like this is for benchmarking disk performance, and is the only reason it exists.

When running an I/O-intensive benchmark, you want to be sure that the various settings you try are all actually doing disk I/O, so Linux allows you to drop caches rather than do a full reboot.

To quote from the documentation:

This file is not a means to control the growth of the various kernel caches (inodes, dentries, pagecache, etc...) These objects are automatically reclaimed by the kernel when memory is needed elsewhere on the system.

Use of this file can cause performance problems. Since it discards cached objects, it may cost a significant amount of I/O and CPU to recreate the dropped objects, especially if they were under heavy use. Because of this, use outside of a testing or debugging environment is not recommended.

share|improve this answer
    
Of course, depending on what you are trying to do, even a full reboot might not sufficiently clear the disk cache. –  Michael Kjörling May 21 at 15:01

The basic idea here is probably not that bad (just very naive and misleading): There may be files being cached, that are very unlikely to be accessed in the near future, for example logfiles. These "eat up" ram, that will later have to be freed when necessary by the OS in one or another way.

Depending on your settings of swappiness, file access pattern, memory allocation pattern and many more unpredictable things, it may happen that when you don't free these caches, they will later be forced to be reused, which takes a little bit more time than allocating memory from the pool of unused memory. In the worst case the swappiness settings of linux will cause program memory to be swapped out, because linux thinks those files may be more likely to be used in the near future than the program memory.

In my environment, linux guesses quite often wrong, and at the start of most europe stock exchanges (around 0900 local time) servers will start doing things that they do only once per day, needing to swap in memory that was previously swapped out because writing logfiles, compressing them, copying them etc. was filling up cache to the point where things had to be swapped out.

But is dropping caches the solution to this problem? definetly not. What would be the solution here is to tell linux what it doesn't know: that these files will likely not be used anymore. This can be done by the writing application using things like posix_fadvise() or using a cmd line tool like vmtouch (which can also be used to look into things as well as cache files).

That way you can remove the data that is not needed anymore from the caches, and keep the stuff that should be cached, because when you drop all caches, a lot of stuff has to be reread from disk. And that at the worst possible moment: when it is needed; causing delays in your application that are noticeable and often unacceptable.

What you should have in place is a system that monitors your memory usage patterns (e.g. if something is swapping) and then analyze accordingly, and act accordingly. The solution might be to evict some big files at the end of the day using vtouch; it might also be to add more ram because the daily peak usage of the server is just that.

share|improve this answer
    
All the apps on my server is running on nohup. Maybe nohup.out is being cached and eating up memory? –  ivcode May 21 at 8:27
    
@ivcode: This could be a reason, check how big nohup.out is. Maybe use vmtouch to figure out how much of it is cached. –  PlasmaHH May 21 at 8:32
    
I have a cron job to cat /dev/null > path/nohup.out in every 15 minutes as nohup.out is growing rapidly. Maybe linux is caching nohup.out even if I'm clearing it –  ivcode May 21 at 8:41
4  
@ivcode If you don't need the output from nohup you should re-direct it to /dev/null. It sounds like you had some very inexperienced sysadmins working on your systems at some point. See stackoverflow.com/questions/10408816/… for how to direct nohup's output to /dev/null –  David Wilkins May 21 at 13:28
    
although nohup.out is cleared in 15 min intervals, if apps process got killed for some reason, nohup.out will be automatically backedup from another script. i tried vmtouch. it's a very good tool indeed –  ivcode May 21 at 15:02

I have seen drop caches to be useful when starting up a bunch of virtual machines. Or anything else that uses Large Pages such as some database servers.

Large Pages in Linux often need to defrag RAM in order to find 2MB of contiguous physical RAM to put into a page. Freeing all of the file cache makes this process very easy.

But I agree with most of the other answers in that there is not a generally good reason to drop the file cache every night.

share|improve this answer
    
I upvoted for pointing out second order prejudice is responses to drop caches. –  Noah Spurrier May 23 at 8:44

It is possible that this was instituted as a way to stabilize the system when there was no one with the skills or experience to actually find the problem.

Freeing resources

Dropping caches will essentially free up some resources, but this has a side effect of making the system actually work harder to do what it is trying to do. If the system is swapping (trying to read and write from a disk swap partition faster than it is actually capable) then dropping caches periodically can ease the symptom, but does nothing to cure the cause.

What is eating up memory?

You should determine what is causing a lot of memory consumption that makes dropping caches seem to work. This can be caused by any number of poorly configured or just plain wrongly utilized server processes. For instance, on one server I witnessed memory utilization max out when a Magento website reached a certain number of visitors within a 15 minute interval. This ended up being caused by Apache being configured to allow too many processes to run simultaneously. Too many processes, using a lot of memory (Magento is a beast sometimes) = swapping.

Bottom Line

Don't just assume that it is something that is necessary. Be proactive in finding out why it is there, have the guts to disable it if others suggest it is wrong, and observe the system - learn what the real problem is and fix it.

share|improve this answer

Linux/m68k actually has a kernel bug which causes kswapd to go crazy and eat up 100% CPU (50% if there’s some other CPU-bound task, like a Debian binary package autobuilder – vulgo buildd – running already), which can (most of the time; not always) be mitigated by running this particular command every few hours.

That being said… your server is most likely not an m68k (Atari, Amiga, Classic Macintosh, VME, Q40/Q60, Sun3) system ;-)

In this case, the person who put in the lines either followed some questionable or, at best, outdated advice, or got the idea about how RAM should be used wrong (modern thinking indeed says “free RAM is RAM wasted” and suggests caching), or “discovered” that this “fixes”[sic!] another problem elsewhere (and was too lazy to search for a proper fix).

share|improve this answer

One reason might be the site is running some kind of monitoring, that checks the amount of free ram and sends a warning to administrators when free ram drops below a certain percentage. If that monitoring tool is dumb enough not to include cache in the free ram calculation, it might send false warnings; regularily emptying the cache could suppress these warnings while still allowing the tool to notice when "real" ram gets low.

Of course, in this kind of situation, the real solution is to modify the monitoring tool to include cache in the free ram calculation; cleaning the cache is just a workaround, and a bad one as well, because the cache will refill quickly when processes access the disk.

So even if my assumption is true, the cache-cleaning is not something that makes sense, it's rather a workaround by someone who isn't competent enough to fix the primary problem.

share|improve this answer

Just to add my two cents: The system knows very well that these memory pages are caches, and will drop as much as needed when an application asks for memory.

A relevant setting is /proc/sys/vm/swappiness, which tells the kernel whether to prefer dropping memory caches vs. swapping "idle" allocated memory pages that are, when new allocations happen.

share|improve this answer

I have a desktop machine with 16GB of RAM running on PAE kernel. After a an hour or two the disk performance degrades dramatically until I drop the caches so I simply put it into cron. I don't know if this is a problem with PAE kernel or with the cache implementation being so slow if there is plenty of memory.

share|improve this answer
6  
This is a prime example of the "cargo cult" system administration: rather than locating and solving the problem, you are simply masking it. –  Michael Hampton May 23 at 13:03

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.