We have production servers running under Debian Linux which host several busy Tomcat instances, database and support services. The systems ran stable for a couple of years, but lately we seem to experience slowdowns and ran into memory issues.
During this time the applications hosted by the Tomcats grew in size, more users, more Tomcat instances. It seems we start to run against the machine‘s memory limits.
I started to familiarize myself with memory monitoring using tools like htop and Java JMX trying to determine the present memory requirements. The identified knobs on the JVM side are switches like for setting heap space maximum and initial size. The memory monitoring parameters are virtual VIRT and reserved memory RES.
My problem is now to find out how much memory we need in the machines, as optimization efforts for the hosted applications might take a while to succeed.
Summing up all virtual sizes gives a multiple of the physical ram and is probably not a good number because the kernel might handle identical parts like common library code.
Summing up all reserved sizes should be close to actual memory usage, less shared memory usage. But it is the result of a dynamic process where memory allocations by the kernel and the different applications and stuff like order of starting the various Tomcat instances might play a role.
Before I now start a bisection trial and error approach, increasing RAM and measuring resulting system performance until we reach calmer waters, I posted this question in the hope that there might be a means to get a better estimate of RAM requirements.
$ cat /proc/meminfo MemTotal: 66075980 kB MemFree: 2117304 kB Buffers: 396328 kB Cached: 9286764 kB SwapCached: 794700 kB Active: 53198584 kB Inactive: 10075240 kB Active(anon): 50010632 kB Inactive(anon): 3587764 kB Active(file): 3187952 kB Inactive(file): 6487476 kB Unevictable: 5604 kB Mlocked: 5604 kB SwapTotal: 4194300 kB SwapFree: 324 kB Dirty: 49460 kB Writeback: 72 kB AnonPages: 52802056 kB Mapped: 89356 kB Shmem: 4448 kB Slab: 388132 kB SReclaimable: 324892 kB SUnreclaim: 63240 kB KernelStack: 11360 kB PageTables: 126924 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 37232288 kB Committed_AS: 47441088 kB VmallocTotal: 34359738367 kB VmallocUsed: 386700 kB VmallocChunk: 34325801336 kB HardwareCorrupted: 0 kB AnonHugePages: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 93868 kB DirectMap2M: 8259584 kB DirectMap1G: 58720256 kB $