6

We are setting up monitoring for GC for Kafka, which is running Jolokia JMX agent. We are using Telegraph to ship JMX out of the system to our Grafana servers for plotting the graph.

When we check the G1 GC count for Old Generation we can see that collection count and collection time is enter image description here

But the drop in memory pool for G1 Old Gen is evident in the following image enter image description here

The Java arguments for running the Kafka process by using the following command:

-javaagent:/usr/lib/jolokia/jolokia-jvm-1.5.0-agent.jar -Xmx2G -Xms2G -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+ExplicitGCInvokesConcurrent -Djava.awt.headless=true -Xloggc:/home/test/kafka_2.12-1.0.1/bin/../logs/kafkaServer-gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=8000 -Djava.rmi.server.hostname=172.31.24.149 -Dkafka.logs.dir=/home/test/kafka_2.12-1.0.1/bin/../logs -Dlog4j.configuration=file:./bin/../config/log4j.properties

Can anybody point out the issue.

Bidyut
  • 933
  • 1
  • 14
  • 30

1 Answers1

10

G1 garbage collector utilizes 3 types of collections (all are Stop-the-World)

  • young GC collects only young space
  • mixer GC collects young space + few regions from old space
  • full GC - emergency GC collecting whole things, it is triggered if incremental collection cannot keep up with application memory usage

Ideally full GC should never happen in G1.

Concerning JMX counters, - young and mixer are counted as young GC - full GC is counted as old

As I said, in case of G1, old GC JMX counter is expected to stay at 0, though old space is reclaimed (incrementally) by young collections.

Alexey Ragozin
  • 8,081
  • 1
  • 23
  • 23