0

Input to my python script is the name of a consumer group and the output is supposed to be the list of topics under the consumer group.
My code currently, doesn't return the topics for which the current offset is -1.
Is there a better way, by which I can obtain the list of all topics under the consumer group using Kafka-python.
I am actually looking to replace describe a topic in a group cmd using Kafka tools.

from kafka import KafkaAdminClient
topics_in_groups = {};
client = KafkaAdminClient(bootstrap_server='localhost:9092')
for group in client.list_consumer_groups():
  topics_in_groups[group[0]] = [];
  topic_dict = client.list_consumer_group_offsets(group[0]);
  for topic in topic_dict:
    topics_in_groups[group[0]].append(topic.topic)
user11322908
  • 23
  • 5

2 Answers2

1

client.list_consumer_group_offsets(group[0]) will get you the metadata of consumer group, the below code will give you the list of topics and partitions of a group.

topics_in_groups = []
for topic,prtitions in topic_dict:
    topics_in_groups.append({"topic":topic, "partition": partition})
Niranjan
  • 137
  • 3
1

I tried this and worked.

from kafka import KafkaAdminClient
topics_in_groups = {}
client = KafkaAdminClient(bootstrap_servers=['broker:9092'])

for group in client.list_consumer_groups():
    topics_in_groups[group[0]] = []

for group in topics_in_groups.keys():
    my_topics = []
    topic_dict = client.list_consumer_group_offsets(group)
    for topic in topic_dict:
        my_topics.append(topic.topic)
        topics_in_groups[group] = list(set(my_topics))

for key , value in topics_in_groups.items():
    print(key, "\n\t", value)
Dheepan Ram
  • 11
  • 2