I have a 8GB RAM linux box on which 4 tomcat servers are running. One of the them is set to 3000MB memory(jvm -Xms and -Xmx setting) and others are set to 1500MB. The swap partition is also set to 8Gigs. When I start these servers, the swap file usage is low. But over a period of days and during certain times when one/all of the servers are in peak activity , the swap usage starts increasing. Here's a typical sar -r output.
kbmemfree kbmemused %memused kbbuffers kbcached kbswpfree kbswpused %swpused kbswpcad
48260 8125832 99.41 196440 2761852 7197688 1190912 14.20 316044
75504 8098588 99.08 198032 2399460 7197688 1190912 14.20 316032
It shows 14.2% swap used currently. The funny thing is this % NEVER decreases. It continues to increase and reach up to 30-40%. We restart our servers weekly.
I would assume the %swpused to increase during periods of peak activity and decrease during periods of low activity..Or at least remain constant. This looks like the swap space is never reclaimed by the OS..
Output of free : free -m total used free shared buffers cached Mem: 7982 7937 45 0 32 2088 -/+ buffers/cache: 5816 2166 Swap: 8191 1163 7028
So there's at least 2g of free Ram. So the question is Why is the swap space continuing to increase and not being reclaimed by the OS ? Or how to debug this to figure out whats hapenning..