2

We got a memory error in our EA and we used -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/path/to/some/dir to dump the heap at the time of OOM.

We had a heap memory of 12GB and perm gen of 256MB.

The heap dump is generated in the Linux box where we have our application running and its size is 11.5GB. We don't have privileges to download it to our local.

When we tried to analyze that 11GB heap dump using JHAT it thrown an OOM.

We tried the following commands from our Linux CLI.

jhat java_pid1491.hprof
jhat -J-Xmx16g -XX:-UseBiasedLocking java_pid1491.hprof
jhat -J-d64 -J-Xmx16g -J-XX:-UseBiasedLocking java_pid1491.hprof#1

For all the options its throwing an OOM exception after reading the dump for several minutes(>30 mins).

We googled it and found MAT as a powerful heap dump analyzer, but not a way to use it in LINUX.

Any suggestions would be of greater help. Thanks.

Modified:

Got MAT installed in Linux x86_64 machine but got following error when executed ./MemoryAnalyzer

(.:17319): GLib-GObject-WARNING **: invalid (NULL) pointer instance
(.:17319): GLib-GObject-CRITICAL **: g_signal_connect_data: assertion `G_TYPE_CHECK_INSTANCE (instance)' failed
(.:17319): Gtk-CRITICAL **: gtk_settings_get_for_screen: assertion `GDK_IS_SCREEN (screen)' failed
(.:17319): GLib-GObject-CRITICAL **: g_object_get: assertion `G_IS_OBJECT (object)' failed
(.:17319): Gtk-WARNING **: Screen for GtkWindow not set; you must always set
a screen for a GtkWindow before using the window
(.:17319): Gdk-CRITICAL **: gdk_pango_context_get_for_screen: assertion `GDK_IS_SCREEN (screen)' failed
(.:17319): Pango-CRITICAL **: pango_context_set_font_description: assertion `context != NULL' failed
(.:17319): Pango-CRITICAL **: pango_context_set_base_dir: assertion `context != NULL' failed
(.:17319): Pango-CRITICAL **: pango_context_set_language: assertion `context != NULL' failed
(.:17319): Pango-CRITICAL **: pango_layout_new: assertion `context != NULL' failed
(.:17319): Pango-CRITICAL **: pango_layout_set_text: assertion `layout != NULL' failed
(.:17319): Pango-CRITICAL **: pango_layout_set_attributes: assertion `layout != NULL' failed
(.:17319): Pango-CRITICAL **: pango_layout_set_alignment: assertion `layout != NULL' failed
(.:17319): Pango-CRITICAL **: pango_layout_set_ellipsize: assertion `PANGO_IS_LAYOUT (layout)' failed
(.:17319): Pango-CRITICAL **: pango_layout_set_single_paragraph_mode: assertion `PANGO_IS_LAYOUT (layout)' failed
(.:17319): Pango-CRITICAL **: pango_layout_set_width: assertion `layout != NULL' failed
(.:17319): Pango-CRITICAL **: pango_layout_get_extents: assertion `layout != NULL' failed
(.:17319): Gtk-CRITICAL **: gtk_icon_theme_get_for_screen: assertion `GDK_IS_SCREEN (screen)' failed
(.:17319): Gtk-CRITICAL **: gtk_settings_get_for_screen: assertion `GDK_IS_SCREEN (screen)' failed
(.:17319): Gtk-CRITICAL **: gtk_icon_size_lookup_for_settings: assertion `GTK_IS_SETTINGS (settings)' failed
(.:17319): Gtk-WARNING **: Invalid icon size 6
(.:17319): Gtk-CRITICAL **: gtk_icon_theme_load_icon: assertion `GTK_IS_ICON_THEME (icon_theme)' failed
Segmentation fault

I tried ./ParseHeapDump.sh ../java_pid1491.hprof this and got following error,

Caused by: java.lang.NoClassDefFoundError: com/ibm/dtfj/image/CorruptDataException
        at org.eclipse.mat.dtfj.InitDTFJ.stop(InitDTFJ.java:70)
raksja
  • 3,969
  • 5
  • 38
  • 44
  • 1
    the first errors are quite normal, since there's no DISPLAY available as you are in SSH. For the second error, perhaps you could find an answer [here](http://www.eclipse.org/forums/index.php?t=msg&goto=652942&S=237ea1900130ae87306793cd0f38015f) – Grooveek May 18 '11 at 09:44
  • 2
    SAP donated MAT code to Eclipse foundation, and provide a [link](http://wiki.eclipse.org/index.php/MemoryAnalyzer#System_Dumps_and_Heap_Dumps_from_IBM_Virtual_Machines) in case of lack of dtfj. Maybe if you install it... – Grooveek May 18 '11 at 09:50
  • I already checked first [link](http://www.eclipse.org/forums/index.php?t=msg&goto=652942&S=237ea1900130ae87306793cd0f38015f) you gave. I replaced the given jars with the feature jars available. It made entire MAT corrupted. Second [link](http://wiki.eclipse.org/index.php/MemoryAnalyzer#System_Dumps_and_Heap_Dumps_from_IBM_Virtual_Machines), I am yet to test and will get back to you. Thanks. – raksja May 18 '11 at 10:47

3 Answers3

2

I was using MAT on Linux yesterday.... check out the proper version here

Grooveek
  • 10,046
  • 1
  • 27
  • 37
  • @Grooveek - I'm downloading the file.. is there any docs for setting up that in linux.. pls share if you have any. thanks. – raksja May 17 '11 at 09:16
  • 1
    that's absolutely straightforward... just cd the install directory and execute `./MemoryAnalyzer`. Then go to `File` and `parse [or load] heap dump` – Grooveek May 17 '11 at 09:25
  • I couldn't execute using `./MemoryAnalyzer`, but I used `./ParseHeapDump.sh ../java_pid1491.hprof` to parse the file. Everything works as expected (parsing, scanning, extracting, Re-indexing, Writing), but after a while it throws `java.lang.NoClassDefFoundError: com/ibm/dtfj/image/CorruptDataException`. I couldn't find any solution googling it. Why it is asking for IBM's tool class when I am trying to run it for my Sun JVM. Pls help me. – raksja May 17 '11 at 11:35
  • what happens when you try to execute `MemoryAnalyzer` ? MAT is a follow-up of an IBM exclusive tool. So, I'm not surprised to see IBM classes in it... Which Linux are you running ? – Grooveek May 17 '11 at 12:21
  • I'm running it in Linux x86_64 using SSH. Error occurred related to GUI settings. If you want full error log, I can post that too. Any clues? – raksja May 18 '11 at 05:42
  • Can't you have X forwarding in SSH to run MAT anyway ? And yes, please post a more consequent stack trace if you can (you can edit your first post, maybe) – Grooveek May 18 '11 at 07:23
  • I have edited the original question and added the exception what i got. Thnks 4 ur help. – raksja May 18 '11 at 08:56
  • MAT is not an follow-up of an IBM exclusive tool. It's a followup of a SAP tool. – kohlerm May 20 '11 at 12:51
  • @kolherm I've seen this while searching information about it, but didn't correct myself in the answer, but in comments to the questions. Thanks for spotting – Grooveek May 20 '11 at 12:57
1

forget Jhat, you would need more than 100Gybte of RAM to parse this dump, it would take forever and it would be very hard to find something. I just lately installed MAT on a SUSE 11 machine. worked flawlessly . Can you please make sure that MAT is using the right JVM? Not sure whether we testet it with anything else then the SUN/SAP JVMs.

Regards, Markus (kohlerm)

kohlerm
  • 2,596
  • 1
  • 17
  • 22
0

We have had success in the past using Netbeans to analyze OOM errors. It seemed to cope where Eclipse and JHAT couldn't.

Fortyrunner
  • 12,702
  • 4
  • 31
  • 54