I'm using Prometheus JMX Exporter to monitor Kafka. I've defined the following pattern rules in the JMX config file:
- pattern : kafka.server<type=(.+), name=(.+)PerSec\w*, topic=(.+)><>Count
name: kafka_server_$1_$2_total
labels:
topic: "$3"
- pattern: kafka.server<type=(.+), name=(.+)PerSec\w*><>Count
name: kafka_server_$1_$2_total
type: COUNTER
- pattern: kafka.server<type=(.+), name=(.+), clientId=(.+), topic=(.+), partition=(.*)><>(Count|Value)
name: kafka_server_$1_$2
labels:
clientId: "$3"
topic: "$4"
partition: "$5"
- pattern: kafka.server<type=(.+), name=(.+), topic=(.+), partition=(.*)><>(Count|Value)
name: kafka_server_$1_$2
labels:
topic: "$3"
partition: "$4"
- pattern: kafka.server<type=(.+), name=(.+), topic=(.+)><>(Count|Value)
name: kafka_server_$1_$2
labels:
topic: "$3"
type: COUNTER
- pattern: kafka.server<type=(.+), name=(.+), clientId=(.+), brokerHost=(.+), brokerPort=(.+)><>(Count|Value)
name: kafka_server_$1_$2
labels:
clientId: "$3"
broker: "$4:$5"
- pattern: kafka.server<type=(.+), name=(.+), clientId=(.+)><>(Count|Value)
name: kafka_server_$1_$2
labels:
clientId: "$3"
- pattern: kafka.server<type=(.+), name=(.+)><>(Count|Value)
name: kafka_server_$1_$2
Now I'm having the following issue. When I send data to the topic in this way:
/bin/kafka-console-producer.sh --broker-list kafka-hostname:9092 --topic test1
The counter of the metric kafka_server_brokertopicmetrics_bytesin_total
increases correctly.
When I try to send data by using the following code:
"use strict";
const envs = process.env;
const options = {
"metadata.broker.list": "kafka-hostname:9092",
"group.id": "kafka1",
topic: "test1",
key: "testKey"
};
const kafkesque = require("untubo")(options);
let count = 0;
const interval = setInterval(function() {
kafkesque.push({ hello: "world", count });
console.log("sent", count);
count++;
}, 500);
process.once("SIGINT", function() {
clearInterval(interval);
console.log("closing");
kafkesque.stop(() => {
console.log("closed");
});
});
In this case the metric doesn't change at all but I can receive the message in the consumer. I think there is something not configured properly in the pattern. Do you have any idea?