10

I have a 8GB RAM linux box on which 4 tomcat servers are running. One of the them is set to 3000MB memory(jvm -Xms and -Xmx setting) and others are set to 1500MB. The swap partition is also set to 8Gigs. When I start these servers, the swap file usage is low. But over a period of days and during certain times when one/all of the servers are in peak activity , the swap usage starts increasing. Here's a typical sar -r output.

kbmemfree kbmemused %memused kbbuffers kbcached kbswpfree kbswpused %swpused kbswpcad

48260 8125832 99.41 196440 2761852 7197688 1190912 14.20 316044

75504 8098588 99.08 198032 2399460 7197688 1190912 14.20 316032

It shows 14.2% swap used currently. The funny thing is this % NEVER decreases. It continues to increase and reach up to 30-40%. We restart our servers weekly.

I would assume the %swpused to increase during periods of peak activity and decrease during periods of low activity..Or at least remain constant. This looks like the swap space is never reclaimed by the OS..

Output of free : free -m total used free shared buffers cached Mem: 7982 7937 45 0 32 2088 -/+ buffers/cache: 5816 2166 Swap: 8191 1163 7028

So there's at least 2g of free Ram. So the question is Why is the swap space continuing to increase and not being reclaimed by the OS ? Or how to debug this to figure out whats hapenning..

Improve this question
0

4 Answers 4

Reset to default
13

If info is swapped out to disc and later read back into memory, it will often be left allocated in the swap area until swap space runs low. That means that if the same info needs to be swapped out again later and hasn't changed, the OS can just drop the pages from allocated RAM without needing to write anything to disc saving time.

Swap allocated to stuff that has been read back into memory will be freed either

  1. when the relevant pages are no longer needed at all (i.e. are freed by the application)
  2. when the relevant pages are changed (so the copy on disc is no longer up to date)
  3. the machine runs low on swap space so clears some things that are already in RAM to make room

Look in /proc/meminfo for a line called "SwapCached". This entry counts pages that are found both in RAM and in swap partitions. For instance, picking a small VM at random, the /proc/meminfo virtual file one of my VMs shows:

SwapTotal:        698816 kB
SwapFree:         624520 kB
SwapCached:        17232 kB

indicating that 74268K of swap space is allocated, but that 17232K worth of those pages are also currently mapped into RAM too (so could be deallocated from swap at a moment's notice if the space is needed by something else).

Also there will no doubt be pages sat there that was swapped out ages ago and have never been used again since. The kernel will not reload pages from swap just because there is some free RAM to read it back into as that free RAM might be better used for cache or buffers - pages written to swap are generally only reread when they are next needed.

If you want to clear out what is in swap, as long as you have enough free and/or freeable (i.e. free+cache+buffers (less those parts of the c+b counts that are not freeable RightThisInstant)), just turn it off and back on again with swapoff -a && swapon -a.

Of course you could also have a memory leak somewhere too, but that is not the only explanation for the behaviour you are seeing.

Improve this answer
1
  • Excellent answer.Thanks. So my system currently shows SwapTotal: 8388600 kB SwapFree: 7197688 kB SwapCached: 595724 kB So Actual Swap Free = 7197688 + 595724 = 7793412. Actul Swap Used = 8388600 - 7793412 = 595188 == 581MB. I guess 581MB of swap file usage is reasonable for a 8GB system with 4 apps running. Let me monitor this for the next couple of days and see if swapCached figure keeps on increasing proportiontate to %swapUsed.
    – Zenil
    Commented Jan 8, 2010 at 10:11
2

Basically, you dont need to care about this. What's important to know is how much IO goes to the swap (have a look at the 'vmstat' command). Having more stuff in the swap doesnt cost anything. The only cost is putting stuff in the swap (page in) or taking it out (page out). So it is perfectly reasonalbe for the OS to let the swap grow.

Improve this answer
1
  • 1
    It is important to know figure out why the swap is growing and never decreasing..What if we allow our servers to continue running for weeks? Will the swap grow beyond the limit and result in memory issues? The swapin/swapout looks "reasonable"..During periods of peak activity there is high swapin/swapout (and also %swapused increases)..During normal periods swapin/swapout is minimal but %swapused is not decreasing
    – Zenil
    Commented Jan 7, 2010 at 15:57
0

as long as you have swap space available, there is no need for the os to free swap space. It will be freed when there is no space left. Bu when you get in this situation you definitely have a problem.

Improve this answer
0

There is no way to tell if this will eventually become an issue unless you run a server long enough to see if it becomes an issue.

Basically the OS will swap things that are not being used out to keep some memory free at all times, in case a new program gets started. Swap space will not be freed until it is needed, meaning you can have 100% swap space used and have no performance issues. The thing to worry about is if this is being caused by a memory leak. It is not necessarily a memory leak but could be.

Java is not prone to memory leaks, but it can happen with complicated apps especially.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .