0

I am running a program which creates an internal data structure for a very large tree. Each node in the tree contains HashMaps containing objects called RHS.

I am trying to debug memory usage using Java VisualVM. I have attached the photo below.

I understand the memory usage of String[], RHS, and Rule because each Rule contains one RHS which in turn contains a String[], hence why there are a similar number of instances of each. The one thing I don't understand is where I'm getting all these Object[] from. I think its also interesting that there are relatively few Object[] but they are each much larger than any of the objects being instantiated explicitly in my code

I never instantiate an Object[] anywhere in my code. Are these created as a byproduct of some other standard library data structure I'm using? (e.g. HashMap, HashSet, etc.)

When I try to heap dump it crashes my computer because the heap is too large. How can I investigate this memory issue more closely?

Java VisualVM Snapshot

EDIT: I ran it on a smaller tree so that I could get a full heapdump, but I am a bit confused by the results.

I have screenshotted below a snapshot and a heap dump taken about 3 seconds apart. They show radically different information about the distribution of memory allocation by class. What does that mean?

Snapshot 2

Heap dump 2

Jeremy Dohmann
  • 1
  • 1
  • Why do you need such a data structure which results in 6 GB heap? – yılmaz Jul 17 '18 at 19:01
  • I suggest you use Flight Recorder to trace allocations. This will show you where they are allocated. Often `Object[]` are the underlying arrays for `ArrayList`. – Peter Lawrey Jul 17 '18 at 20:04
  • A lot of collection classes wrap an object array, most notably, `ArrayList` wraps an `Object[]`, for example. And I suppose, your nodes have to keep track of their children, somehow. Besides that, your first dump indicates an impact of a Serialization operation which does not appear in the other dumps. Serialization keeps track of all encountered objects during an operation, stored in some sort of custom hash set implementation backed by a large `Object[]` array. – Holger Jul 17 '18 at 20:04
  • @Holger could you explain a little bit more about your comment on Serialization? – Jeremy Dohmann Jul 17 '18 at 20:06
  • In which regard? As said, it keeps track of all encountered objects, so when encountering an object again, it stores a back-reference to it, so it can restore an identical object graph on the other side, which implies tracking of all restored objects on the other side too. – Holger Jul 17 '18 at 20:10
  • Due to erasure, all parametrized classes like `Map` are actually just like `Map`. A `HashMap` uses big arrays, a `HashSet` is just a `HashMap` with dummy values, etc. Make sure, you close your `Object*Stream`, otherwise it'll keep references until GC'ed. – maaartinus Jul 17 '18 at 20:38
  • @maaartinus that makes sense, thank you, what do you mean by Object * Stream? – Jeremy Dohmann Jul 18 '18 at 16:58
  • @Holger Ok, I understand now, thank you. – Jeremy Dohmann Jul 18 '18 at 16:58
  • @JeremyDohmann As Holger wrote, serialization keeps track of all objects passed through, so when you forget to close (or reset?) the `Object*Stream`, they'll hold a lot of references. – maaartinus Jul 18 '18 at 17:46

0 Answers0