Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

We have a java application running on two RHEL5.5 systems. We recently got into a situation where we needed to add more memory for both systems.

Each system was rebooted within 5 minutes of each other. We confirmed that the systems were even in connections through the our load balance device. The free output looked like the following:


             total       used       free     shared    buffers     cached
Mem:       3977340    3570688     406652          0      26472    3194816
-/+ buffers/cache:     349400    3627940
Swap:      2097144          0    2097144


             total       used       free     shared    buffers     cached
Mem:       3977340    1369456    2607884          0      44200     860736
-/+ buffers/cache:     464520    3512820
Swap:      1048568          0    1048568

While I would expect a difference in the memory currently being used for cache, the extreme difference seems rather disconcerting. Is there any method to see which files currently have blocks in cache or any other way to determine why such a large difference would be evident in two systems that are mirror clones of each other, in a load-balanced setup with relatively close reboot times?

I realize that they stems are not in a bad state, however, I'm being asked to provide a reason or explanation as to why one system is bringing so much into cache and the other is not.

Other VM settings such as swappiness, min_free_kbytes and such are all equal.

Any ideas on what steps I would take to figure this out?

share|improve this question
What algorithm is used for the load balancing? – Ignacio Vazquez-Abrams Nov 2 '10 at 1:40
It uses a 'cyclic' algorithm ( in radware terms ). i.e, round-robin. – sjmh Nov 2 '10 at 2:49
like other have said, it maybe the fact the one machine is getting more requests than the other, check etstat -s| grep active on both machines, and see if they is a great difference. – The Unix Janitor Nov 2 '10 at 13:09

You can use lsof to look for files being used by both Java process on both machine, that might help in pointing towards problem.

Also it would be nice idea to verify on the system where used memory and corresponding cached memory is more that the memory being used/cached is actually by your java program and not some other program.

share|improve this answer

Compare the process lists with ps aux, top or whatever method you like. In addition to that see a breakdown of kernel memory usage with slabtop and compare the output of those.

OK, with RHEL 5.5 you're most likely using ext3 as your file system, but for example XFS is heavily utilizing free memory and that becomes very evident with slabtop, but can leave you confused if you are just staring free and other memory reports.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.