1

All,

I have a requirement to expose the Apache Kafka metrics to the spring boot (v 2.3.0.RELEASE) actuator endpoint.

Please note, I am NOT using spring-kafka library .

I am using the following libraries

        <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>kafka-streams</artifactId>
            <version>2.5.0</version>
        </dependency>


        <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>kafka-clients</artifactId>
            <version>2.5.0</version>
        </dependency>

I have tried this spring.jmx.enabled=true but seems like this doesnt work. I assume this is happening because spring is not managing the kafka.

Is there a way I can bind these JMX metrics to the micrometer MeterRegistry?

I am able to make this work using the jmx-exporter provided by prometheus, but since that requires an agent running on a different port, I was hoping to make this work with default Micrometer and spring boot.

Rahul Kumar
  • 87
  • 1
  • 12

1 Answers1

2

Have you tried KafkaClientMetrics, KafkaStreamsMetrics or KafkaConsumerMetrics? KafkaConsumerMetrics collects metrics through JMX but it is deprecated in favor of KafkaClientMetrics.

You only need to create them and bind to your registry:

KafkaConsumer<String, String> consumer;
KafkaProducer<String, String> producer;
MeterRegistry registry;

...

new KafkaClientMetrics(consumer).bindTo(registry);
new KafkaClientMetrics(producer).bindTo(registry);

Similarly for KafkaStreams:

new KafkaStreamsMetrics(kafkaStreams).bindTo(registry);

If Spring does not manage your Kafka components, why do you expect spring.jmx.enabled=true to do anything?

Jonatan Ivanov
  • 4,895
  • 2
  • 15
  • 30
  • 1
    Oh, i forgot about this thread. (I believe we also had a discussion on this on micrometer issues section). So, I was able to resolve this issue. Since my code uses Kstreams, the above configuration doesn't work directly. In order to achieve this with Kstreams, there should be a public method annotated with ```EventListener(ContextRefreshedEvent.class)``` and inside that I get the streams instance from ```StreamsBuilderFactoryBean.getKafkaStreams()```... Once thats done, i create an object of KafkaStreamsMetrics using the streams object and then bind it to meterRegistry – Rahul Kumar Apr 01 '21 at 05:33
  • @RahulKumar you save my day! – vyacheslav.kislov Jun 10 '21 at 21:13
  • @RahulKumar can you pls share the sample code on how to see Kafka producer metrics using Spring boot – Manish Kumar Jul 13 '21 at 10:47