How to find the processing time of Kafka messages?

Question

I have an application running Kafka consumers and want to monitor the processing time of each message consumed from the topic. The application is a Spring boot application and exposes Kafka consumer metrics to Spring Actuator Prometheus endpoint using micrometre registry. Can I use kafka_consumer_commit_latency_avg_seconds or kafka_consumer_commit_latency_max_seconds to monitor or alert?

Answer 1

Those metrics have nothing to do with record processing time. spring-kafka provides metrics for that; see here.

Monitoring Listener Performance

Starting with version 2.3, the listener container will automatically create and update Micrometer Timer s for the listener, if Micrometer is detected on the class path, and a single MeterRegistry is present in the application context. The timers can be disabled by setting the ContainerProperty micrometerEnabled to false.

Two timers are maintained - one for successful calls to the listener and one for >failures.