How to find the byte rate of Consumers Kafka

Question

I created an application which use Kafka. So how can I find how many MB/sec my consumers reads ?

My topics have only one partition

score 0 · Answer 1 · answered Nov 24 '20 at 14:42

0

Are you using the Java Kafka consumer API? If yes, some JMX metrics are exposed by it and some more specific to consumer fetching. You can see them here: https://kafka.apache.org/documentation/#consumer_fetch_monitoring

answered Nov 24 '20 at 14:42

ppatierno

I try to find that via cli with kafka.tools.JmxTool or Jconsole but when I open Jconsole I don't find smth about that. Also when I use kafka.tools.JmxTool I take some errors – Apostolos Lazidis Nov 24 '20 at 14:45
if you run your application setting this env var you should be able to see metrics `KAFKA_JMX_OPTS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false "` This is actually what the kafka-console-consumer script does for example. If you want to get metrics from inside your code, the `KafkaConsumer` should have a `metrics` method for this. – ppatierno Nov 24 '20 at 15:55
I defined `KAFKA_JMX_OPTS` in my kafka container. Should I define `KAFKA_JMX_OPTS` and in env of my application? I have access at broker metrics with `Jconsole` and `kafka.tools.JmxTool`. Also is there any way to take the consumer metrics with `kafka.tools.JmxTool`? – Apostolos Lazidis Nov 24 '20 at 16:33

1 Answers1