14

I have a linux box that is used as SAN storage. It is equipped with 14 SATA HDDs and the only software it run is iscsi target and zabbix agent. This is physical box, not VM.

The RAM size installed is 128 GB, swap was arranged of 4 GB and no-one ever care for its size when RAM is quite big, but as the box run (the uptime is 694 days) swap was used more and more, and now it used for ~80%.

The htop won't show anything bad, looks like OS uses RAM as cache etc. swappiness was set to 10, then to 1, none helped. Surely I can disable swap completely but looks like this will be bad move.

So my question is simple, how can I set the box to use RAM and not use swap, and why it uses swap at all while RAM is enough?

This is SAN, so it serves blocks not files, so cache can be done on block bases, and can easily be reduced if RAM is limited, it won't impact the logic of iSCSI anyway.

Zabbix says right now I have RAM available = 123.22 GB (hey, almost all of 128 GB!), swap used is 3.24 GB of 4 GB.

8
  • 2
    Do you see significant memory usage outside of fs cache? What does free -m tell you? Any processes using a lot of memory (top, sort by memory M; ps auxf, htop). There's paged out memory in swap, but is the system actively swapping? What does vmstat 60 tell you throughout the hours?
    – marcelm
    Mar 11, 2020 at 17:35
  • 1
    How do you determine swap is being used? It might simply be saving memory pages to disk as part of preparing to use the corresponding ram for something else. Mar 11, 2020 at 22:21
  • Could it be for limiting power usage? Keeping stuff in RAM needs energy where it's free on disk (minus the access cost). I'm not sure whether linux can actually enable/disable the power supply to the RAM though.
    – Richard
    Mar 12, 2020 at 0:43
  • 1
    IIRC ZFS uses a lot of memory; and its memory usage is not readily apparent from top and family. Are you using ZFS?
    – muru
    Mar 12, 2020 at 7:30
  • 1
    Are you troubleshooting a performance issue? If so, please share those details. If not, I suspect you're seeing a problem where none exists. RAM and swap are resources available to the operating system. The fact that both are highly utilized means the OS is getting maximum benefit from both resources.
    – JakeRobb
    Mar 12, 2020 at 14:26

1 Answer 1

23

There is no problem. Even if there is still available RAM, the Linux Kernel will move memory pages which are hardly ever used into swap space.

It’s good to swap out memory pages that have been inactive for a while, keeping often-used data in cache; this is the desired situation of the Kernel.

You can have more control on this by using vfs_cache_pressure (which controls the tendency of the kernel to reclaim the memory which is used for caching of directory and inode objects) and swappiness (which controls how aggressive the kernel will swap memory pages: higher values will increase aggressiveness, lower values decrease the amount of swap).

You can find these settings in /etc/sysctl.conf.

10
  • 11
    To elaborate: there's some things that are kept in RAM, but seldom or never used. Think of cron, or the initializing part of some other daemons. I'd rather want them in swap, freeing a few memory pages for active jobs, than using RAM itself. Swapping out doesn't immediately remove them from RAM; it just means the kernel can drop them whenever it feels something else has better use for RAM. So as long as RAM is plenty, having some pages in swap doesn't hurt; when RAM becomes scarce, having pages in swap helps. Mar 11, 2020 at 16:36
  • 4
    -1: This answer is mostly based on myths. Linux shouldn't use swap when there's more than about 3% of RAM available (or even less for systems with a lot of RAM). More on that here: unix.stackexchange.com/questions/533739/…
    – Spc_555
    Mar 11, 2020 at 17:05
  • 5
    @Spc_555 I think that question's definition of "opportunistic swapping" is different from the one needed here. That question asks whether pages can be swapped out when the system is doing nothing. Apparently, they can't. But this question asks whether anonymous (non-file-backed) pages can be swapped out, to make more space for disk caches. Apparently they can.
    – user253751
    Mar 11, 2020 at 18:18
  • 4
    @Spc_555 If that were true, then why does every major Linux distro in the history of forever behave the way this answer describes? Are they all misconfigured out of the box? Mar 11, 2020 at 23:02
  • 7
    @AlexeyKamenskiy It heavily depends on the amount of swapping. Swapping per se doesn't decrease host performance if it is a "one-time" action. If the same pages have to be constantly swapped in and out, that's what decreases performance.
    – glglgl
    Mar 12, 2020 at 10:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.