Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am hoping to find some good methods (by looking at /proc) or good tools that I can use to determine the effectiveness of the disk cache. I would like to be able to determine just how much of the RAM used in the disk cache is being actively used. I am hoping to get a better understanding of the disk cache usage, so that I can more accurately purchase RAM for future servers.

Ideally, something I'd like to see (though I don't expect this level of detail), would be some report like this (or a method for making this report myself):

1.4GB of RAM is used in the disk cache which is used for 10 IO requests per second
2.3GB of RAM "" "" for 2 requests per second
5.3GB of RAM "" "" is basically never used
share|improve this question

1 Answer 1

up vote 1 down vote accepted

I don't think it's possible to gather information like this. Memory isn't really used to process requests. There may only be one request to map a page in memory and then it may remain used for long periods of time without the system knowing how much it's used except for the periodic probing to look for stale memory.

Also, the reasoning behind the request is misguided. Even if most of the cache is never used, the hit rate of the part that is used is still determined by how much cache you have.

Let me give you analogy. Say there are 20 different kinds of balls and you can keep 18 of them in stock. If you cache randomly, there's an 18/20 chance you'll have the ball someone wants in stock. So if someone asks you for a particular ball and you have it in stock, only 1/18 of your balls will have been used. But you'll have a 18/20 chance of having that ball because of the other 17 balls sitting there unused.

So amount used to serve requests isn't really the right measurement.

The system doesn't really keep enough information to allow you to figure out how well the cache would have done had it been a different size.

Update: Let me try one more time to explain why this won't work. You're trying to infer from the fact that 5GB of RAM was used for cache that wasn't accessed that the system would have performed roughly the same if it had had 5GB less RAM. But that's totally false.

Say you run a book store. You notice that in a given month, you only sell 10% of your inventory. You think: "What a waste. 90% of my books are just sitting there unused. I don't need to keep so much inventory." So you reduce your inventory by 90%. What do you think will happen?

Yes, after the fact you see that lots of your inventory didn't sell. But with a much smaller inventory, most of your customers wouldn't have found the book they wanted in inventory. Knowing after the fact which books didn't sell doesn't mean you could have made do with a smaller inventory before the fact -- before you knew which books people would want and which they wouldn't.

So even if the information you want was available, it wouldn't let you draw the conclusion you want to draw. You would have to keep enough information to run a simulation and say -- had I kept fewer things in memory, which would I have kept? And thus, would I still have had the information that later got used?

So even if only a small amount of the data in the cache winds up being used, unless you could have predicted which information would be used, you can't infer that the rest of the cache didn't have a significant performance impact by allowing you to keep in memory the data that was used.

share|improve this answer
    
Maybe my example output for such a program was confusing. I meant to suggest that in the example above, the system would have a total of 9GB of RAM. –  GoldenNewby Apr 21 '12 at 18:10
    
Right, but the fact that 5.36GB of RAM wasn't used doesn't mean that the system wouldn't have been much slower if it had 5.36GB less RAM. See my update. –  David Schwartz Apr 21 '12 at 20:01
    
I went ahead and accepted your answer, but really it was rather disappointing. It feels like in your book store analogy, the book store should be able to remove books from the inventory if they are unpopular. Additionally, it seems reasonable that both the book store and Linux would ask people what book they were looking for, even if it isn't in inventory. It sounds to me like Linux just simply doesn't keep track of how popular certain cached requests are. –  GoldenNewby Apr 24 '12 at 20:17
    
@GoldenNewby: The vast majority of books are unpopular. But having so many unpopular books makes a great bookstore. But you are correct that Linux could, in principle, keep sufficient information to simulate the affects of different cache sizes. However, that's really much more complicated than it sounds. (Think about the bookstore trying to figure out which books it would have decided to carry and not carry in the past.) –  David Schwartz Apr 24 '12 at 23:05

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.