How does my jre exceed its heap limits?

Question

I have a jvm whose actual memory use is well in excess of the limits with which it's being launched. It's a Sun VM:

root@jira:/opt/atlassian/jira/jre# ./bin/java -version
java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot(TM) Server VM (build 20.1-b02, mixed mode)

and it's been configured with a max heap of 768MB (-Xmx768m) and a max permgen of 256MB (-XX:MaxPermSize=256m) but the processes is going well past 1GB, currently at 1.7GB.

I'm used to Java apps with memory leaks killing themselves with OutOfMemoryErrors when they exceed their limits, not being killed externally when the Linux OOMKiller starts shooting processes.

From ps:

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
jira     23030  2.7 70.5 2181356 1457748 ?     Sl   Nov30  34:52 /opt/atlassian/jira/jre//bin/java -Djava.util.logging.config.file=/opt/atlassian/jira/conf/logging.properties -XX:MaxPermSize=256m -Xms256m -Xmx768m

There's 1.5GB and it's going to keep growing until the 2GB box starts to swap.

What am I misunderstanding about JVM memory limits?

Answer 1

The heap limit is the limit of the JVM heap presented to Java applications (no Java program can allocate more than that amount of space on the heap), but does not limit the size of the Java process (which includes the JVM heap that you're limiting, plus any items allocated by the JVM itself, plus the stack (for the JVM and your Java application), plus other stuff I'm almost certainly leaving out.

Normally you won't see much more than 10-15% bloat over the limits you set, but pathological cases probably exist, and there may even be a memory leak or other bug in the java runtime itself (good luck debugging that pig).

If you want to limit the size of the Java runtime at the OS level you should probably look in to the ulimit command. The JVM may handle hitting those limits gracefully.

Answer 2

There's the off heap area which includes memory allocated by JNI code and Direct Byte buffers. You can control the last using -XX:MaxDirectMemorySize.