How to list all producers of a kafka cluster?

Question

I am able to list all Kafka Consumer with KafkaAdminClient:

AdminClient client = AdminClient.create(conf);
ListTopicsResult ltr = client.listTopics();
KafkaFuture<Set<String>> names = ltr.names();

ArrayList<ConsumerGroupListing> consumerGroups = new ArrayList<>(client.listConsumerGroups().all().get());
ConsumerGroupListing consumerGroup = consumerGroups.get(0);

Is it possible to list all registrated producers in a similar way?

Giorgos Myrianthous · Accepted Answer · 2020-02-20T11:50:44.413

9

In contrast to consumers, it is not possible to retrieve such information since Kafka brokers don't store any kind of information about the producers connected to them.

edited Feb 20 '20 at 11:50

answered Feb 20 '20 at 11:37

Giorgos Myrianthous

36,235
20
134
156

Have you got a reference for this? – Samuel Åslund Sep 13 '21 at 08:31
2

This is not true as the producers will be registered upon the first write and their ids expire by default 7 days after the last write. See https://docs.confluent.io/platform/current/installation/configuration/broker-configs.html#brokerconfigs_transactional.id.expiration.ms. – Andras Hatvani Nov 08 '22 at 11:43

score 0 · Answer 2 · answered Jul 21 '23 at 19:29

It is possible to get all active producer using AdminClient API:

Collection<TopicPartition> topicPartitions = ...

DescribeProducersResult result = adminClient.describeProducers(
  Collections.singleton(topicPartition)
);

Map<TopicPartition, DescribeProducersResult.PartitionProducerState> producerStateMap = result.all().get();

https://kafka.apache.org/32/javadoc/org/apache/kafka/clients/admin/KafkaAdminClient.html#describeProducers(java.util.Collection,org.apache.kafka.clients.admin.DescribeProducersOptions)

How to list all producers of a kafka cluster?

2 Answers2