3

Why ZooKeeper was replaced with kafka-raft for metadata management in Apache kafka ?

I went through the articles on apache's blog ( apache.org ) as well, but couldn't find exact and concise reasons.

Looking forward to some good insights.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Amit kumar
  • 2,169
  • 10
  • 25
  • 36
  • What would you suggest it is replaced with instead? Do you want to manage Zookeeper cluster **and** Kafka, or just Kafka? – OneCricketeer Jul 22 '22 at 16:34
  • I am not suggesting something to replace zookeeper with. I am just trying to figure, what was the reason to deprecate zookeeper and choose kafka raft instead ? – Amit kumar Jul 23 '22 at 04:30
  • Primarily because it'll make Kafka easier to install, compared to other products like NATS, RabbitMQ, ActiveMQ, etc that don't require external resources – OneCricketeer Jul 24 '22 at 05:14

1 Answers1

4

From the Kafka Improvement Proposal (KIP) document:

Currently, Kafka uses ZooKeeper to store its metadata about partitions and brokers, and to elect a broker to be the Kafka Controller. We would like to remove this dependency on ZooKeeper. This will enable us to manage metadata in a more scalable and robust way, enabling support for more partitions. It will also simplify the deployment and configuration of Kafka.

Read more here: KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum

Robin Moffatt
  • 30,382
  • 3
  • 65
  • 92