4

My JVM is basically a spark executor which is running tasks one after another. A task is memory hungry and requires significant memory during its lifecycle.

JConsole and JVisualVM report side by side

The above JVM is running on G1GC with default params. As you can see in the VisualVM report on the right hand side between 4:25 to 4:32 PM, the spikes are due to each tasks run by the executor (essentially, each spike is due to the executor picking up new task after the previous one is finished). When I triggered a manual GC at 4:35, I saw a sharp decline the heap usage. Also, as you can see on the left hand side in the JConsole report, the old gen space is never collected by G1GC (the sharp decline in old gen space just before 16:35 is due to the manual GC).

As my application is a spark batch job application, I am ok if the JVMs spent good amount of time doing GC. But, I am running bit short on memory. So, I wanted to know how I can tune my JVM G1GC params so that there are more frequent GC (with the old gen space also getting collected) and I can get the work done with considerably lesses heap space (XMX).

akshay
  • 49
  • 5
  • have you check this:`https://www.oracle.com/technetwork/articles/java/g1gc-1984535.html#Taming` – ZhaoGang Dec 25 '18 at 10:56
  • I tried changing -XX:InitiatingHeapOccupancyPercent . I tried with 0, 5, 10. Lower the value, more frequent was the GC and more CPU it would consume, but the behaviour would not be consistent. After sometime, the heap usage would spike up and if we have set the XMX value lesser (6 GB) than what's there in the picture above, it would throw OOM – akshay Dec 25 '18 at 11:10

1 Answers1

2

For G1, promptly returning unused memory to the operating system is an OpenJDK 12 feature:

If you a short on memory, it's likely that you need to keep RSS (total memory allocated from the operating system) under control, not just used memory in the Java heap. With G1 in current OpenJDK, (semi-)manually triggered full GCs appear to be the only way to trigger RSS reduction. (G1 will perform poorly if you change the ergonomics that full GCs happen frequently.)

If your OpenJDK build contains the Shenondoah collector, it may be a better choice if you need to conserve memory, particularly with some tuning, such as using -XX:ShenandoahGCHeuristics=compact.

ZGC should eventually support returning memory as well, but it currently does not; the patch adding an -XX:+ZReleaseUnusedHeap option has not been merged.

Florian Weimer
  • 32,022
  • 3
  • 48
  • 92
  • that `ShenandoahGCHeuristics` will perform back-to-back GC calls without delay, at least `ShenandoahAllocationThreshold` should also be enabled. – Eugene Jan 18 '20 at 19:05