Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

In our servers we have a habit of dropping caches at midnight.

sync; echo 3 > /proc/sys/vm/drop_caches

When I run the code it seems to free up lots of RAM, but do I really need to do that. Isn't free RAM a waste?

share|improve this question
51  
Find the person who put this in and ask him why he did it. As you correctly guessed, there is no obvious good reason for it. –  Michael Hampton May 20 '14 at 3:15
9  
Debugging the kernel. That's about it. This doesn't actually free up any RAM; it drops caches, as the name suggests, and thus reduces performance. –  Michael Hampton May 20 '14 at 3:38
26  
@ivcode Then you should find and fix the problem with that server rather than trying to avoid the conditions that cause it. If my car stalled every time I made a sharp right turn, avoiding sharp right turns is a lousy fix. –  David Schwartz May 20 '14 at 5:01
6  
Related thedailywtf.com/Articles/Modern-Memory-Management.aspx Strongly arguing it's a bad idea. –  Drunix May 20 '14 at 9:22
6  
Related, and a useful description of the "problem": linuxatemyram.com –  Bill Weiss May 20 '14 at 16:47

11 Answers 11

You are 100% correct. It is not a good practice to free up RAM. This is likely an example of cargo cult system administration.

share|improve this answer
7  
+1 for mentioning Cargo Cult System Administration. Any sysadmin who doesn't know that term and what it means should be fired. –  Tonny May 20 '14 at 10:22
8  
@Tonny: We would be left without sysadmin department then :( –  PlasmaHH May 20 '14 at 19:44
4  
Surely it's at least plausible that one could fully see and agree such practices are bad without knowing that specific disparaging term for them? I guess arguably, any sysadmin who doesn't spend time on the kind of forums where people are continually criticising cargo-cult practices as such, isn't keeping their knowledge current ;-) –  Steve Jessop May 23 '14 at 10:55
1  
Like most of humanity, I love terse brash assertions with lots of approval, but a cite or reasoning would earn my superego's +1. –  Aaron Hall May 23 '14 at 20:22
1  
Explain the cargo-cult administration, as well as the above, if you don't mind. Maybe in a follow-on edit? I'm still withholding my +1... :P –  Aaron Hall May 26 '14 at 2:56

Yes, clearing cache will free RAM, but it causes the kernel to look for files on the disk rather than in the cache which can cause performance issues.

Normally the kernel will clear the cache when the available RAM is depleted. It frequently writes dirtied content to disk using pdflush.

share|improve this answer
15  
+1 for explaining why it's a bad idea. –  Ogre Psalm33 May 20 '14 at 17:00

The reason to drop caches like this is for benchmarking disk performance, and is the only reason it exists.

When running an I/O-intensive benchmark, you want to be sure that the various settings you try are all actually doing disk I/O, so Linux allows you to drop caches rather than do a full reboot.

To quote from the documentation:

This file is not a means to control the growth of the various kernel caches (inodes, dentries, pagecache, etc...) These objects are automatically reclaimed by the kernel when memory is needed elsewhere on the system.

Use of this file can cause performance problems. Since it discards cached objects, it may cost a significant amount of I/O and CPU to recreate the dropped objects, especially if they were under heavy use. Because of this, use outside of a testing or debugging environment is not recommended.

share|improve this answer
    
Of course, depending on what you are trying to do, even a full reboot might not sufficiently clear the disk cache. –  Michael Kjörling May 21 '14 at 15:01
    
"these objects are automatically reclaimed by the kernel when memory is needed" is the design goal but it might not always be the actual behavior. –  Dan Pritts Jan 14 at 15:51
    
@DanPritts What precisely makes you think it's not so? –  Joe Jan 28 at 3:24
    
The obvious case is when you want to clear out RAM to allow the allocation of more (non-trnsparent) hugepages; another case is transparent hugepage garbage collection pause bugs (see my answer/comments elsewhere on this question). But my comment was intended for the general case. Sometimes the people who are operating the system know better than the people who designed/implemented it. Often, not - that's what their comment is trying to protect against. I'm just glad that the –  Dan Pritts Jan 28 at 15:24

The basic idea here is probably not that bad (just very naive and misleading): There may be files being cached, that are very unlikely to be accessed in the near future, for example logfiles. These "eat up" ram, that will later have to be freed when necessary by the OS in one or another way.

Depending on your settings of swappiness, file access pattern, memory allocation pattern and many more unpredictable things, it may happen that when you don't free these caches, they will later be forced to be reused, which takes a little bit more time than allocating memory from the pool of unused memory. In the worst case the swappiness settings of linux will cause program memory to be swapped out, because linux thinks those files may be more likely to be used in the near future than the program memory.

In my environment, linux guesses quite often wrong, and at the start of most europe stock exchanges (around 0900 local time) servers will start doing things that they do only once per day, needing to swap in memory that was previously swapped out because writing logfiles, compressing them, copying them etc. was filling up cache to the point where things had to be swapped out.

But is dropping caches the solution to this problem? definetly not. What would be the solution here is to tell linux what it doesn't know: that these files will likely not be used anymore. This can be done by the writing application using things like posix_fadvise() or using a cmd line tool like vmtouch (which can also be used to look into things as well as cache files).

That way you can remove the data that is not needed anymore from the caches, and keep the stuff that should be cached, because when you drop all caches, a lot of stuff has to be reread from disk. And that at the worst possible moment: when it is needed; causing delays in your application that are noticeable and often unacceptable.

What you should have in place is a system that monitors your memory usage patterns (e.g. if something is swapping) and then analyze accordingly, and act accordingly. The solution might be to evict some big files at the end of the day using vtouch; it might also be to add more ram because the daily peak usage of the server is just that.

share|improve this answer
    
All the apps on my server is running on nohup. Maybe nohup.out is being cached and eating up memory? –  ivcode May 21 '14 at 8:27
    
@ivcode: This could be a reason, check how big nohup.out is. Maybe use vmtouch to figure out how much of it is cached. –  PlasmaHH May 21 '14 at 8:32
    
I have a cron job to cat /dev/null > path/nohup.out in every 15 minutes as nohup.out is growing rapidly. Maybe linux is caching nohup.out even if I'm clearing it –  ivcode May 21 '14 at 8:41
4  
@ivcode If you don't need the output from nohup you should re-direct it to /dev/null. It sounds like you had some very inexperienced sysadmins working on your systems at some point. See stackoverflow.com/questions/10408816/… for how to direct nohup's output to /dev/null –  David Wilkins May 21 '14 at 13:28
    
although nohup.out is cleared in 15 min intervals, if apps process got killed for some reason, nohup.out will be automatically backedup from another script. i tried vmtouch. it's a very good tool indeed –  ivcode May 21 '14 at 15:02

I have seen drop caches to be useful when starting up a bunch of virtual machines. Or anything else that uses Large Pages such as some database servers.

Large Pages in Linux often need to defrag RAM in order to find 2MB of contiguous physical RAM to put into a page. Freeing all of the file cache makes this process very easy.

But I agree with most of the other answers in that there is not a generally good reason to drop the file cache every night.

share|improve this answer
    
I upvoted for pointing out second order prejudice is responses to drop caches. –  Noah Spurrier May 23 '14 at 8:44

It is possible that this was instituted as a way to stabilize the system when there was no one with the skills or experience to actually find the problem.

Freeing resources

Dropping caches will essentially free up some resources, but this has a side effect of making the system actually work harder to do what it is trying to do. If the system is swapping (trying to read and write from a disk swap partition faster than it is actually capable) then dropping caches periodically can ease the symptom, but does nothing to cure the cause.

What is eating up memory?

You should determine what is causing a lot of memory consumption that makes dropping caches seem to work. This can be caused by any number of poorly configured or just plain wrongly utilized server processes. For instance, on one server I witnessed memory utilization max out when a Magento website reached a certain number of visitors within a 15 minute interval. This ended up being caused by Apache being configured to allow too many processes to run simultaneously. Too many processes, using a lot of memory (Magento is a beast sometimes) = swapping.

Bottom Line

Don't just assume that it is something that is necessary. Be proactive in finding out why it is there, have the guts to disable it if others suggest it is wrong, and observe the system - learn what the real problem is and fix it.

share|improve this answer

Linux/m68k actually has a kernel bug which causes kswapd to go crazy and eat up 100% CPU (50% if there’s some other CPU-bound task, like a Debian binary package autobuilder – vulgo buildd – running already), which can (most of the time; not always) be mitigated by running this particular command every few hours.

That being said… your server is most likely not an m68k (Atari, Amiga, Classic Macintosh, VME, Q40/Q60, Sun3) system ;-)

In this case, the person who put in the lines either followed some questionable or, at best, outdated advice, or got the idea about how RAM should be used wrong (modern thinking indeed says “free RAM is RAM wasted” and suggests caching), or “discovered” that this “fixes”[sic!] another problem elsewhere (and was too lazy to search for a proper fix).

share|improve this answer

One reason might be the site is running some kind of monitoring, that checks the amount of free ram and sends a warning to administrators when free ram drops below a certain percentage. If that monitoring tool is dumb enough not to include cache in the free ram calculation, it might send false warnings; regularily emptying the cache could suppress these warnings while still allowing the tool to notice when "real" ram gets low.

Of course, in this kind of situation, the real solution is to modify the monitoring tool to include cache in the free ram calculation; cleaning the cache is just a workaround, and a bad one as well, because the cache will refill quickly when processes access the disk.

So even if my assumption is true, the cache-cleaning is not something that makes sense, it's rather a workaround by someone who isn't competent enough to fix the primary problem.

share|improve this answer

Just to add my two cents: The system knows very well that these memory pages are caches, and will drop as much as needed when an application asks for memory.

A relevant setting is /proc/sys/vm/swappiness, which tells the kernel whether to prefer dropping memory caches vs. swapping "idle" allocated memory pages that are, when new allocations happen.

share|improve this answer

I can think of one plausible reason to do this in a nightly cron job.

On a large system, it may be useful to periodically drop caches so you can remove memory fragmentation.

The kernel transparent hugepage support does a periodic sweep of memory to coalesce small pages into hugepages. Under degenerate conditions this can result in system pauses of a minute or two (my experience with this was in RHEL6; hopefully it's improved). Dropping caches may let the hugepage sweeper have some room to work with.

You might argue that this is a good reason to disable transparent hugepages; OTOH you may believe that the overall performance improvement from transparent hugepages is worth having, and worth paying the price of losing your caches once a day.

share|improve this answer
1  
Do you have any source for this? This sounds like something that should be fixed in the kernel if it's such an issue. –  gparent Jan 14 at 15:59
    
I have personal experience with the pauses with transparent hugepages. RHEL6, Dell R810, 4CPUs, 64GB RAM. Disabling transparent hugepages (there's a /proc file to do so) immediately fixed the pauses. I didn't try the cache drop technique at the time; instead I reconfigured our java apps to use non-transparent hugepages, and left transparent hugepages disabled. IIRC, we looked into the situation enough to realize that we weren't the only people affected, and that Red Hat knew about the issue. –  Dan Pritts Jan 14 at 16:07

I have a desktop machine with 16GB of RAM running on PAE kernel. After a an hour or two the disk performance degrades dramatically until I drop the caches so I simply put it into cron. I don't know if this is a problem with PAE kernel or with the cache implementation being so slow if there is plenty of memory.

share|improve this answer
9  
This is a prime example of the "cargo cult" system administration: rather than locating and solving the problem, you are simply masking it. –  Michael Hampton May 23 '14 at 13:03
1  
Sometimes the expedient solution is the right one. It might just be putting off resolving the real problem, or it might be as much solution as is required in the circumstances. Even if it's bad practice, it's still not "cargo cult." There's a demonstrated cause and effect: drop caches and disk performance improves. –  Dan Pritts Jan 14 at 15:49

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.