0

With Python-kafka admin client I know how to create and delete topics.

But is there any way to check if that topic exists or not? If not then only I wanted to create it. How can I do that in python -kafka admin client? I didnt see anything in the documentation.

1689
  • 117
  • 1
  • 10
  • you can use kafka script ? kafka-topic.sh --list – maxime G Aug 18 '20 at 08:33
  • i need python script – 1689 Aug 18 '20 at 10:05
  • Idk but you can probably call `kafka-topic.sh --list` from any python script and then parse the output. Or better use `kafkaConsumer.topics()` ref: https://kafka-python.readthedocs.io/en/1.0.2/apidoc/KafkaConsumer.html#kafka.KafkaConsumer.topics – WiLL_K Aug 18 '20 at 11:32
  • I am not sure how to call kafka-topic.sh --list in python script. Can you just tell me that? – 1689 Aug 19 '20 at 06:26

2 Answers2

0

Create a KafkaConsumer object without specifying the optional input argument topics and then call the .topics() on the created object.

krishnatg
  • 1
  • 1
0

If you try to create topic that is already exist then Kafka admin client raise Exception TopicAlreadyExistsError so you can handle that exception in your code like I did...

Let me provide you the sample code I built for my use case

def create_topic(self, topic_name=None, num_partitions=1, replication_factor=1):
    try:
        topic_list = []
        topic_list.append(NewTopic(name=topic_name, num_partitions=num_partitions, replication_factor=replication_factor))
        self.admin_client.create_topics(new_topics=topic_list, validate_only=False)
        self.show_topic_on_console()
        return True
    except TopicAlreadyExistsError as err:
        print(f"Request for topic creation is failed as {topic_name} is already created due to {err}")
        return False
    except Exception as err:
        print(f"Request for topic creation is failing due to {err}")
        return False

Alternative flow - you can design like below

  1. Extract list of the topics
  2. Check whether request topic in the list or not
  3. If not in the list the create topic
Abhishek Jain
  • 3,815
  • 2
  • 26
  • 26