Java on Linux insufficient memory even though there is plenty of available memory being used for caching

Question

I’m trying to start a process on Red Hat Enterprise Linux Server release 6.5 (Santiago), which is failing because it is not able to allocate sufficient memory.

# There is insufficient memory for the Java Runtime Environment to continue. 
# Cannot create GC thread. Out of system resources.
# An error report file with more information is saved as:

The problem is not with the Java application itself - I get the error too when I simply run:

java -version

Looking at memory consumption:

free -m

             total       used       free     shared    buffers     cached
Mem:         32069      31276        792          0        556      16948
-/+ buffers/cache:      13771      18297
Swap:         9325         38       9287

So Mem is reporting that 31 out of 32 Gb of memory is being used, but there should be 18Gb of memory available that is being used by the cache. Should the OS not free up some of this when a process requests memory?

The details of the error dump is

#
# There is insufficient memory for the Java Runtime Environment to continue.
# Cannot create GC thread. Out of system resources.
# Possible reasons:
#   The system is out of physical RAM or swap space
#   In 32 bit mode, the process size limit was hit
# Possible solutions:
#   Reduce memory load on the system
#   Increase physical memory or swap space
#   Check if swap backing store is full
#   Use 64 bit Java on a 64 bit OS
#   Decrease Java heap size (-Xmx/-Xms)
#   Decrease number of Java threads
#   Decrease Java thread stack sizes (-Xss)
#   Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
#  Out of Memory Error (gcTaskThread.cpp:46), pid=40816, tid=140071992215296
#
# JRE version: 6.0_26-b03
# Java VM: Java HotSpot(TM) 64-Bit Server VM (20.1-b02 mixed mode linux-amd64 compressed oops)

---------------  T H R E A D  ---------------

Current thread (0x00007f6508006800):  JavaThread "Unknown thread" [_thread_in_vm, id=40817,              stack(0x00007f650d46c000,0x00007f650d56d000)]

Stack: [0x00007f650d46c000,0x00007f650d56d000],  sp=0x00007f650d56b7c0,  free space=1021k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x85ebd5]  VMError::report_and_die()+0x265
V  [libjvm.so+0x3e41b8]  report_vm_out_of_memory(char const*, int, unsigned long, char const*)+0x68
V  [libjvm.so+0x466ada]  GCTaskThread::GCTaskThread(GCTaskManager*, unsigned, unsigned)+0x13a
V  [libjvm.so+0x4655de]  GCTaskManager::initialize()+0x21e
V  [libjvm.so+0x465373]  GCTaskManager::GCTaskManager(unsigned)+0x13
V  [libjvm.so+0x722cad]  ParallelScavengeHeap::initialize()+0x4dd
V  [libjvm.so+0x836549]  Universe::initialize_heap()+0xa9
V  [libjvm.so+0x8360ea]  universe_init()+0x7a
V  [libjvm.so+0x4ac53b]  init_globals()+0x4b
V  [libjvm.so+0x81cc74]  Threads::create_vm(JavaVMInitArgs*, bool*)+0x214
V  [libjvm.so+0x51a7b0]  JNI_CreateJavaVM+0x80

Answer 1

Have a look into the jvm min max memory, do you use any web server? Look into the number of treads, memory settings for it- for Tomcat most of them are @server.xml. Look into Java Mission Control http://www.oracle.com/technetwork/java/javaseproducts/mission-control/java-mission-control-1998576.html to get extra insight. Linux usually keeps more memory to be ready when requested. For Java on Linux only something like JMC can help you get an idea on what's going on. Have a look in the standard output from the JVM.

Answer 2

Seems like the issue has nothing to do with memory, but with process limits. The user I'm running the process as had a really low process limit set to 1024.

ulimit -u
1024

and the thread count of all the proceses run by this user is close to that:

ps -eLf | grep 'myuser' | wc -l
1022

Answer 3

I think we missed another restriction which will also block the new thread creation, this is the kernel.pid_max limit.

root@myhost:~# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.7 LTS
Release:    16.04
Codename:   xenial
root@myhost:~# uname -a
Linux myhost 4.4.0-190-generic #220-Ubuntu SMP Fri Aug 28 23:02:15 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

In my system, this restriction is the lowest restrition and thus it is always the firstly restricted to be reached and becomes the root cause of my issue.

I find that at least in my system, this threshold kernel.pid_max is 32768. When I launch any simple JVM process, it reports error like below:

java/jstack/jstat ...
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Cannot create GC thread. Out of system resources.
# An error report file with more information is saved as:
# /root/hs_err_pid1390.log

Check the memory, sufficient.

root@lascorehadoop-15a32:~# free -mh
              total        used        free      shared  buff/cache   available
Mem:           125G         11G         41G        1.2G         72G        111G
Swap:            0B          0B          0B

Check the system thread:

~# ps -eLf|wc -l
31506

But I check the system limit by ulimit:

root@myhost:~# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 515471
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 98000
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 515471
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

From the ulimit output, we could see that currently thread number is far less than the maximum user process limit.

In fact, the limit which is reached is the kernel.pid_max

Very easy to check and tuning it: https://www.cyberciti.biz/tips/howto-linux-increase-pid-limits.html

Answer 4

"free -g" will give you the current memory usage, if you are not having enough memory, then increase the physical RAM or increase the swap space.

if you have enough memory, then the next step would be increasing the max user process count.

change ulimit configuration(add below lines at the end)

vi /etc/security/limits.conf

* soft nproc 65535

* hard nproc 65535

* soft nofile 65535

* hard nofile 65535

vi /etc/security/limits.d/90-nproc.conf * soft nproc 65535

* hard nproc 65535

* soft nofile 65535

* hard nofile 65535