1

I have set up a confluent cloud multizone cluster and it got created with just one bootstrap server. There was no setting for choosing number of servers while creating the cluster. Even after creation, I can’t edit the number of bootstrap servers.

I want to know how to increase the number of servers in confluent cloud kafka cluster.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
sd99
  • 13
  • 4

1 Answers1

3

Under the hood, the Confluent Cloud cluster is already running multiple brokers. Depending on your cluster configuration (specifically, whether you're running Standard or Dedicated, and what region and cloud you're in), the cluster will have between six and several dozen brokers.

The way a Kafka client bootstrap server config works is that the client reaches out to the bootstrap server and requests a list of all brokers, and then uses those broker endpoints to actually produce/consume from Kafka (reference: https://jaceklaskowski.gitbooks.io/apache-kafka/content/kafka-properties-bootstrap-servers.html)

In Confluent Cloud, the provided bootstrap server is actually a load balancer in front of all of the brokers; when the client connects to the bootstrap server it'll receive the actual endpoints for all of the actual brokers, and then use that for subsequent connections.

So TL;DR, in your client, you only need to specify the one bootstrap server; under the hood, the Kafka client will connect to the (many) brokers running in Confluent Cloud, and it should all just work.

Source: I work at Confluent.

justin
  • 46
  • 1
  • Thanks a lot for the quick response! It's super helpful!! – sd99 Jul 01 '21 at 11:50
  • 1
    Just curious, is this mentioned somewhere in the docs? Would be great to have it documented if not already there :) – sd99 Jul 01 '21 at 12:00