How to perform bulk topic deletion in kafka

Question

We have 3 node Kafka cluster with replication factor 3 and default log retention period of 4 days. We have enabled the auto topic creation and It will create topics for each of our tasks. Once the task is finished we don't need the topic created for it. Now we have 2000 unused topics there in Kafka, can we perform bulk deletion of this 2000 topics without shutting down our cluster ?. Do bulk topic deletion affects our cluster's performance?. If yes what is the safe way to delete unwanted topics without shutting down cluster

Answer 1

Look into the kafka-topics script from your install folder:

Put the list command into a list(so you can later use it in your favorite language)

list topics

bin/kafka-topics.sh --zookeeper localhost:2181 --list

For each topic from the above list, run the delete command:

delete topic

bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic mytopic

Yes, performance will be affected depending on:

• How many messages you have in a topic and overall

• How big is the message size

Please be carefull and only delete a couple of them at a time at most, then verify if the cluster has finished the delete operations afterwards :)