1

I'm having a Kafka cluster running on Confluent Cloud but I'm not able to reset the commit offset from the UI. Hence, I'm trying to do it via Kafka's CLI as below:

kafka-consumer-groups --bootstrap-server=my_cluster.confluent.cloud:9092 --list

However, I'm bumping into the below error. And I think it has to do with how I can authenticate.

Error: Executing consumer group command failed due to org.apache.kafka.common.KafkaException: Failed to find brokers to send ListGroups
java.util.concurrent.ExecutionException: org.apache.kafka.common.KafkaException: Failed to find brokers to send ListGroups
    at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396)
    at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073)
    at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:165)
    at kafka.admin.ConsumerGroupCommand$ConsumerGroupService.listConsumerGroups(ConsumerGroupCommand.scala:203)
    at kafka.admin.ConsumerGroupCommand$ConsumerGroupService.listGroups(ConsumerGroupCommand.scala:198)
    at kafka.admin.ConsumerGroupCommand$.run(ConsumerGroupCommand.scala:70)
    at kafka.admin.ConsumerGroupCommand$.main(ConsumerGroupCommand.scala:59)
    at kafka.admin.ConsumerGroupCommand.main(ConsumerGroupCommand.scala)
Caused by: org.apache.kafka.common.KafkaException: Failed to find brokers to send ListGroups
    at org.apache.kafka.clients.admin.KafkaAdminClient$24.handleFailure(KafkaAdminClient.java:3368)
    at org.apache.kafka.clients.admin.KafkaAdminClient$Call.handleTimeoutFailure(KafkaAdminClient.java:838)
    at org.apache.kafka.clients.admin.KafkaAdminClient$Call.fail(KafkaAdminClient.java:804)
    at org.apache.kafka.clients.admin.KafkaAdminClient$TimeoutProcessor.handleTimeouts(KafkaAdminClient.java:934)
    at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.timeoutPendingCalls(KafkaAdminClient.java:1013)
    at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.processRequests(KafkaAdminClient.java:1367)
    at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:1331)
    at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: findAllBrokers
OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
knl
  • 969
  • 11
  • 35

2 Answers2

3

here is an example to list consumer groups

kafka-consumer-groups --bootstrap-server <ccloud kafka>:9092 --command-config consumer.properties --list

consumer.properties

bootstrap.servers=<ccloud kafka>:9092
ssl.endpoint.identification.algorithm=https
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule 
required username="<KEY>" password="<SECRET>";
Niranjan
  • 137
  • 3
1

You'll want to use the --command-config option to set properties files that contain your CCLoud credentials

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
  • thanks! Do you know if I can reset offset on a topic for a consumer group with Confluent's CLI? Or should I authenticate then use Apache Kafka's command line tools? Thanks! – knl Feb 15 '22 at 17:36
  • 1
    I don't use the Confluent ccloud cli, but I don't think it has that functionality – OneCricketeer Feb 16 '22 at 14:27
  • Yeap, I think so, thanks! – knl Feb 16 '22 at 17:16
  • I am exactly trying to do the same , trying to offset to a given value . @knl , did you have any luck with this ? – Srikant Barik Feb 17 '22 at 14:32
  • 1
    @Sri That'd should be easier done in the code itself with a seek call given a TopicPartition +Offset – OneCricketeer Feb 17 '22 at 15:05
  • Ok , I figured out the command . Thanks for the help. @OneCricketeer . I have another problem in which I have 30+ consumers in a consumer group , and it is constantly rebalancing, can we get to know for which consumer rebalance is triggered from command line ? – Srikant Barik Feb 17 '22 at 15:50
  • @Sri I'm not familiar with that. You'll need a client log aggregation utility to order them all by timestamp and see which one caused the problem. – OneCricketeer Feb 17 '22 at 16:55