4

I'm using Prometheus JMX Exporter to monitor Kafka. I've defined the following pattern rules in the JMX config file:

- pattern : kafka.server<type=(.+), name=(.+)PerSec\w*, topic=(.+)><>Count
      name: kafka_server_$1_$2_total
      labels:
        topic: "$3"
    - pattern: kafka.server<type=(.+), name=(.+)PerSec\w*><>Count
      name: kafka_server_$1_$2_total
      type: COUNTER
    - pattern: kafka.server<type=(.+), name=(.+), clientId=(.+), topic=(.+), partition=(.*)><>(Count|Value)
      name: kafka_server_$1_$2
      labels:
        clientId: "$3"
        topic: "$4"
        partition: "$5"
    - pattern: kafka.server<type=(.+), name=(.+), topic=(.+), partition=(.*)><>(Count|Value)
      name: kafka_server_$1_$2
      labels:
        topic: "$3"
        partition: "$4"
    - pattern: kafka.server<type=(.+), name=(.+), topic=(.+)><>(Count|Value)
      name: kafka_server_$1_$2
      labels:
        topic: "$3"
      type: COUNTER
    - pattern: kafka.server<type=(.+), name=(.+), clientId=(.+), brokerHost=(.+), brokerPort=(.+)><>(Count|Value)
      name: kafka_server_$1_$2
      labels:
        clientId: "$3"
        broker: "$4:$5"
    - pattern: kafka.server<type=(.+), name=(.+), clientId=(.+)><>(Count|Value)
      name: kafka_server_$1_$2
      labels:
        clientId: "$3"
    - pattern: kafka.server<type=(.+), name=(.+)><>(Count|Value)
      name: kafka_server_$1_$2

Now I'm having the following issue. When I send data to the topic in this way:

/bin/kafka-console-producer.sh --broker-list kafka-hostname:9092 --topic test1

The counter of the metric kafka_server_brokertopicmetrics_bytesin_total increases correctly.

When I try to send data by using the following code:

"use strict";

const envs = process.env;

const options = {
  "metadata.broker.list": "kafka-hostname:9092",
  "group.id": "kafka1",
  topic: "test1",
  key: "testKey"
};

const kafkesque = require("untubo")(options);

let count = 0;
const interval = setInterval(function() {
  kafkesque.push({ hello: "world", count });
  console.log("sent", count);
  count++;
}, 500);

process.once("SIGINT", function() {
  clearInterval(interval);
  console.log("closing");
  kafkesque.stop(() => {
    console.log("closed");
  });
});

In this case the metric doesn't change at all but I can receive the message in the consumer. I think there is something not configured properly in the pattern. Do you have any idea?

Mazzy
  • 13,354
  • 43
  • 126
  • 207
  • Hi Mazzy, can you look at this question and see if you can help? https://stackoverflow.com/questions/58599405/kafka-and-jmx-exporter – ankit patel Oct 29 '19 at 18:12

0 Answers0