Is there a way to do a live heap dump with ibm-jdk for linux?

Question

I know that it's possible to dump heap when an OutOfMemoryException is occuring on this JVM but is it possible to ask a live dump with tools like jmap or jconsole?

The purpose is to find a leak that only happens in a production environment situation. I can't stop the jvm nor wait for an OutOfMemoryException to occur (too long) — Opty, Dec 10 '10 at 07:37

score 6 · Answer 1 · answered Oct 01 '12 at 18:40

You need to be aware that there are "system" dumps (basically OS core files) and "heap" aka portable heap dumps (PHD). The later ones are less usefull as they do not contain actual data. They are hower enabled by default.

On AIX or Linux Typically you will set up -Xdump:system (short for -Xdump:system:events=gpf+user) to allow kill -3 <pid> to trigger a heap dump.

BTW, you can with the default options use kill -ABRT <pid>. However this will terminate your JVM.

Run java -Xdump:what to see your defaults, like:

> /usr/java6/bin/java -Xdump:what -version

Registered dump agents
----------------------
-Xdump:system:
    events=gpf+abort+traceassert,
    label=/home/u0002824/core.%Y%m%d.%H%M%S.%pid.%seq.dmp,
    range=1..0,
    priority=999,
    request=serial
----------------------
...
java version "1.6.0"
Java(TM) SE Runtime Environment (build pap3260sr9fp2-20110627_03(SR9 FP2))

With turned on system dumps:

> /usr/java6/bin/java -Xdump:system -Xdump:what -version

Registered dump agents
----------------------
-Xdump:system:
    events=gpf+user+abort+traceassert,
    label=/home/u0002824/core.%Y%m%d.%H%M%S.%pid.%seq.dmp,
    range=1..0,
    priority=999,
    request=serial
----------------------
-Xdump:heap:
    events=systhrow,
    filter=java/lang/OutOfMemoryError,
    label=/home/u0002824/heapdump.%Y%m%d.%H%M%S.%pid.%seq.phd,
    range=1..4,
    priority=500,
    request=exclusive+compact+prepwalk,
    opts=PHD
----------------------
-Xdump:java:
    events=gpf+user+abort+traceassert,
    label=/home/u0002824/javacore.%Y%m%d.%H%M%S.%pid.%seq.txt,
    range=1..0,
    priority=400,
    request=exclusive+preempt
----------------------
-Xdump:java:
    events=systhrow,
    filter=java/lang/OutOfMemoryError,
    label=/home/u0002824/javacore.%Y%m%d.%H%M%S.%pid.%seq.txt,
    range=1..4,
    priority=400,
    request=exclusive+preempt
----------------------
-Xdump:snap:
    events=gpf+abort+traceassert,
    label=/home/u0002824/Snap.%Y%m%d.%H%M%S.%pid.%seq.trc,
    range=1..0,
    priority=300,
    request=serial
----------------------
-Xdump:snap:
    events=systhrow,
    filter=java/lang/OutOfMemoryError,
    label=/home/u0002824/Snap.%Y%m%d.%H%M%S.%pid.%seq.trc,
    range=1..4,
    priority=300,
    request=serial
----------------------
...

Do not forget to run jre/bin/jextract on the core. * .dmp files.

score 4 · Answer 2 · answered Dec 10 '10 at 09:36

4

You have a few options:

This list isn't exhaustive.

answered Dec 10 '10 at 09:36

McDowell

107,573
31
204
267

Tried JConsole and HotSpot Diagnostic MBean on this JVM and there is no instrumentation operation available to dump heap (maybe it is available for win32 but not on linux) – Opty Dec 10 '10 at 11:46
Regarding the previous comment, the only method available to generate live dump seems to configure dump on sigusr (events=user) using -Xdump on jvm startup. The problem with that is that uncontrolled SIGUSR may happen from time to time on the JVM... – Opty Dec 10 '10 at 11:52
@Opty - you don't say much about the product you're using. See here for some alternatives for generating system dumps: http://publib.boulder.ibm.com/infocenter/javasdk/v6r0/topic/com.ibm.java.doc.diagnostics.60/diag/tools/javadump_trigger.html Note some of the restrictions on Linux: http://publib.boulder.ibm.com/infocenter/javasdk/v6r0/topic/com.ibm.java.doc.diagnostics.60/diag/tools/dumpagents_platform_nonzos.html I would check what that does to memory consumption of a test system under load before trying in production. – McDowell Dec 11 '10 at 17:32
It is a big server application from our company, and we were unable to reproduce the problem under test environment. We just know that it is a supposed memory leak and that it takes a long time to grow (about 1Gb/Month) on a very very fast server with a lot of various user requests. Btw, we suspect a problem directly in IBM's classes since we already found other problems (for ex : never try to add or remove handlers to Logger while actually logging or you may get ArrayIndexOutOfBound exceptions) that SUN JDK didn't have. Btw, we may also use SUN JDK, but it is 20-30% slower... – Opty Dec 11 '10 at 21:26

score 3 · Accepted Answer · answered Dec 11 '10 at 21:42

3

OK, I'll finally answer to myself : the application has a remote admin interface, so I will implement a new command that is calling the com.ibm.jvm.Dump.HeapDump() method.

answered Dec 11 '10 at 21:42

Opty

504
5
10

score 0 · Answer 4 · answered Dec 10 '10 at 07:10

0

I think there is one tool like JProfiler . it 'll nicely work with Eclipse

answered Dec 10 '10 at 07:10

Ravi Parmar

1,392
5
24
46

That would suppose that I need to profile the app... Furthermore I would use the TPTP profiling tool for Eclipse. I need to dump in a live production environment. – Opty Dec 10 '10 at 07:52

Is there a way to do a live heap dump with ibm-jdk for linux?

4 Answers4