Stack Overflow
Stack Overflow

Questions tagged [kafka-exporter]

Kafka Exporter is an open source project to enhance monitoring of Apache Kafka brokers and clients.

Kafka Exporter is provided with AMQ Streams for deployment with a Kafka cluster to extract additional metrics data from Kafka brokers related to offsets, consumer groups, consumer lag, and topics.

The metrics data is used, for example, to help identify slow consumers.

Lag data is exposed as Prometheus metrics, which can then be presented in Grafana for analysis.

If you are already using Prometheus and Grafana for monitoring of built-in Kafka metrics, you can configure Prometheus to also scrape the Kafka Exporter Prometheus endpoint.

Additional resources

Kafka exposes metrics through JMX, which can then be exported as Prometheus metrics.

9 questions
1
vote
1 answer

Strimzi Kafka + Kafka Exporter : consumer group not showing up in Prometheus metric kafka_consumergroup_lag

I've a Strimzi Kafka cluster on GKE, and I've kafkaExporter deployed as well. Kafka Topic has acls, and a Consumer Group (spark-kafka-source-*) defined which can read data from the topic. I'm running a Spark StructuredStreaming programs which reads…
Karan Alang
  • 869
  • 2
  • 10
  • 35
1
vote
0 answers

Kafka exporter for latest kafka (kafka 17)

I want to add monitoring of my kafka. I've found product named kafka exporter which depends on 14.9.3. I use kafka 17(https://artifacthub.io/packages/helm/bitnami/kafka/17.0.0) and it doesn't work with latest kafka…
gstackoverflow
  • 36,709
  • 117
  • 359
  • 710
0
votes
1 answer

Kafka-exporter deployment with custom configmap

I want to deploy Kafka-exporter using helm with custom configmap, my values.yaml file looks like this: # kafkaServer: # - kafka0.svc.cluster.local:9092 # - kafka1.svc.cluster.local:9092 # - kafka2.svc.cluster.local:9092 service: …
sledge13
  • 3
  • 2
0
votes
0 answers

Prometheus query to get the consumer lag increase in the last 5 minutes

I'm using Kafka exporter to monitor the Kafka metrics which is then queried in prometheus. I have a Kafka topic with 3 consumer groups, these 3 consumer groups are used by 3 different services. I am trying to write a query to have an alert when…
perplexedDev
  • 857
  • 4
  • 17
  • 49
0
votes
0 answers

Monitoring apache-Kafka lag using Prometheus. Kafka_exporter or linkedin Burrow

I am trying to monitor Kafka consumer group or topic lag. I want to fire an alert every time the lag of a topic doesn't reduce for 30 minutes or so and only keeps on increasing. I have been trying to use kafka_exporter. And have also tried Linkedin…
Utkarsh Kalra
  • 3
  • 1
0
votes
1 answer

how to collect a metric kafka_log_log_size?

metric kafka_log_log_size in grafana(datasource victoria metrics) I assembled metrics by kafka exporter(https://github.com/danielqsj/kafka_exporter) but metric kafka_log_log_size not going to. how to build such a metric?
Iceforest
  • 309
  • 1
  • 11
0
votes
1 answer

Dashboard visualizing CPU usage of kafka container chopped up

I want to monitor the cpu usage of kafka container, but the graph is chopped up into different pieces. There seem to be gaps in the graph and after each gap a different colored line follows. The time range is last 30 days. For the exporter we use…
Pavol Krajkovič
  • 505
  • 1
  • 12
0
votes
2 answers

How to configure kafka chart to expose both: kafka-metrics and kafka-jmx-metrics to prometheus?

We use kafka chart (version 17) as a dependency of our chart. And for I try to configure monitoring of that kafka. values.yaml: kafka: metrics: kafka: enabled: true jmx: enabled: true serviceMonitor: enabled: true …
gstackoverflow
  • 36,709
  • 117
  • 359
  • 710
-1
votes
1 answer

Error Init Kafka Client: kafka: client has run out of available brokers to talk to: dial tcp: lookup kafka on 10.96.0.10:53: no such host

C:\kafka>kubectl logs kafka-exporter-745f574c74-tzfn4 I0127 12:39:08.113890 1 kafka_exporter.go:792] Starting kafka_exporter (version=1.6.0, branch=master, revision=9d9cd654ca57e4f153d0d0b00ce36069b6a677c1) F0127 12:39:08.890639 1…
Vijay Khanna
  • 1
  • 1