Consider the following:
jstat -gcutil -t $JAVA_PID
Timestamp S0 S1 E O P YGC YGCT FGC FGCT GCT
71704.4 99.98 99.99 100.00 46.51 67.64 47878 5313.498 449 50.385 5363.883
71706.7 99.96 99.96 100.00 47.41 67.64 47884 5314.933 449 50.385 5365.318
71708.8 99.98 99.98 100.00 47.94 67.64 47888 5316.234 449 50.385 5366.619
We have our java server running with: -server -Xms12G -Xmx12G -XX:MaxPermSize=1536M -XX:MaxPermSize=1536M -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:+PrintGCDetails -XX:+PrintGCTimeStamps
From the above TGC column it appears we have 10 Young Garbage Collections hitting over a 4.4 second interval. In addition these collections account for a total of 2.736 seconds of "JVM time" (YGC 5316.234 - 5313.498 = 2.736). This means 2.736/4.4 = 62% of the 4.4 seconds is spent "collecting garbage"
Even though this is a "minor" or young GC event the JVM still pauses for a short time for each of the ten causing a degradation in performance. REF the following excellent article: http://blog.dynatrace.com/2011/03/10/major-gcs-separating-myth-from-reality/
Obviously we have a HUGE heap but in this case the problem does not lie in the "old" gen but rather the young gen, which is quite small. I've always been told adjusting the JVM memory ratios is a bit of a 3rd rail and to avoid if possible, but in this case it seems to perhaps make sense.
Another jstat shows S0 and S1 at only 4MB and Eden at just over 50MB.
jstat -gcnew -t 11795
Timestamp S0C S1C S0U S1U TT MTT DSS EC EU YGC YGCT
85897.4 4224.0 4224.0 0.0 4224.0 1 4 2112.0 34112.0 9933.1 52557 6030.234
Does anyone have any thoughts on optimizing this (other than tweaking code). I though perhaps increasing eden 2 to 4 time in size might help.
What about G1? Would that be an option on the latest version of Java 6? I know it is still considered experimental until Java 7.
Any other thoughts?