How to define new metrics for custom Processor (and make them available in jconsole)?

Question

i have a processor which should produce kstream JMX metrics:

public class ProcessorJMX implements Processor<String, GenericRecord> {
  private StreamsMetrics streamsMetrics;
  private Sensor sensorStartTs;

  @Override
  public void init(ProcessorContext processorContext) {
    streamsMetrics = processorContext.metrics();
    sensorStartTs = streamsMetrics.addSensor("start_ts", Sensor.RecordingLevel.INFO);
  } 
  @Override
  public void process(String key, GenericRecord val) {
    streamsMetrics.recordThroughput(sensorStartTs, Long.valueOf(val.get("start_ts").toString()));
  }
  @Override
  public void punctuate(long l) { }

  @Override
  public void close() { }
}

then i use this on my output topic and start my integration test. but when i look in jconsole, i dont see this metric anywhere. where can i find it in jconsole under MBeans?

do i have to do something else before it becomes visible?

here are the properties i am using:

Properties testProperties = new Properties();
testProperties.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, 
CLUSTER.bootstrapServers());
testProperties.put("confluent.metrics.reporter.bootstrap.servers", CLUSTER.bootstrapServers());
testProperties.put("metrics.recording.level", "DEBUG");
testProperties.put("metric.reporters", "org.apache.kafka.common.metrics.JmxReporter");

what is wrong with this config?

https://docs.confluent.io/current/streams/monitoring.html#built-in-metrics — Matthias J. Sax, Feb 12 '18 at 18:19
hi, how do i "register the metrics reporter to JmxReporter in the configs"? i have added the following: Properties testProperties = new Properties(); testProperties.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, CLUSTER.bootstrapServers()); testProperties.put("confluent.metrics.reporter.bootstrap.servers", CLUSTER.bootstrapServers()); testProperties.put("metrics.recording.level", "DEBUG"); testProperties.put("metric.reporters", "org.apache.kafka.common.metrics.JmxReporter"); did i miss out something? — timmy_stapler, Feb 14 '18 at 13:16
That looks appropriate, you have already specified the reporters in testProperties.put("metric.reporters", ...); Looking at your code closely again, I realized you may forget to add the metric into your sensor, i.e. you need to call `sensorStartTs.add(metricName, MeasurableStat)` where `MeasurableStat` defines the type of the stat, like Sum, Avg, Count, etc. This is because you used the general `addSensor` API to add the sensor; if you used advanced `addThroughputSensor` etc it will call `sensor.add` for you. — Guozhang Wang, Feb 14 '18 at 22:10

score 3 · Accepted Answer · edited May 22 '19 at 10:23

The following is what I added to the init:

@Override
public void init(ProcessorContext processorContext) {
    streamsMetrics = processorContext.metrics();

    Map<String, String> metricTags = new HashMap<String, String>();
    metricTags.put("metricTagKey", "metricsTagVal");

    MetricConfig metricConfig = new MetricConfig().tags(metricTags);
    Metrics metrics = new Metrics(metricConfig);
    sensorStartTs = metrics.sensor("start_ts");
    MetricName metricName = metrics.metricName("x-name", "x-group", "x-description");
    sensorStartTs = streamsMetrics.addSensor("start_ts", Sensor.RecordingLevel.INFO);
    sensorStartTs.add(metricName, new Min());
}

This MetricName class helped.

How to define new metrics for custom Processor (and make them available in jconsole)?

1 Answers1