1

In managing a server with 512GB of RAM I encountered a process that keeps consuming swap until it hits 100% of the swap space then stops consuming more (6GB of swap) but keeps working fine (albeit, when a request enters the process, it takes a long time (20+ min) to get to the required performance).

Even setting swappiness to 0 doesn't prevent this proces from swapping.

Swapping happens while free shows this:

# free -h --giga
              total        used        free      shared  buff/cache   available
Mem:           515G         16G        2.3G         30M        497G        496G
Swap:          5.8G        1.0M        5.8G

The process in question:

# smem -s swap -t -n -k
  PID User     Command                         Swap      USS      PSS      RSS
(...)
36776 1000     java -XX:+UseG1GC -Xms1G -X     1.6M    13.4G    13.4G    13.4G
-------------------------------------------------------------------------------
  148 11                                       2.0M    15.1G    15.2G    15.9G

It keeps growing and growing over time (at a rate of ~20 MB / hour) until 100% swap is consumed. It might be worth mentioning it is running in a docker container but I don't know if that influences anything.

Swappiness:

# cat /proc/sys/vm/swappiness
0

I really want to disable swap completely at this point but this answer strongly recommends against it. What are my options to keep this programs' memory completely in RAM?

3

Since your question is about process inside Docker container, it is worth checking if you are not missing vm.overcommit_memory=1 configuration as described here: Node using swap memory instead of host memory

By default, Docker recommends using a value of vm.swappiness=0 for Docker environments, which prevents swapping except in the case of an OOM (OutOfMemory) condition. All nodes must set vm.overcommit_memory=1, which tells the kernel to always allow memory allocations until there is no truly memory. This article explains a situation that can occur whenever a value other 0 is used for vm.swappiness. If vm.swappiness is set to a value higher than 0, you might notice that only swap memory is being used on the node even though host memory was available.

  • however your swap should be incraise regardless because ususally you takt at least 50 percent of RAM, in your case in my point you should have st least 50/100gb swap – djdomi Aug 11 '19 at 20:18
  • This solution did slow down the increase of the swap "eating" but did not completely stop it. For now my only solution seems to disable swap completely as I cannot find any good reasons to keep swap enabled on a 512GB RAM system. My reason: of those 512GB there is 5 GB truly 'free/unused/wasted' by the Linux Kernel. The swapping has a negative performance impact in the long run on a proces that must be responsive. Out of the 512GB only 30GB's are truly "used up", the rest is cache/buffers/shared. Swapping is also very slow, in the ranges of kilobytes to 2megabytes/s, not 500MB/s (SSD). – Gizmo Aug 11 '19 at 20:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.