KRaft is an Apache Kafka runtime mode that uses the Raft protocol internally between Controller and Broker servers rather than relying on an external Apache Zookeeper installation.
Questions tagged [kraft]
37 questions
1
vote
1 answer
node.id vs broker.id in KRaft mode
The default config for running Kafka in KRaft mode (in config/kraft/server.properties) does not have broker.id. Instead it has node.id. What is the difference between these fields? Confluent says to set broker.id.generation.enable=true when not…
user2233706
- 6,148
- 5
- 44
- 86
0
votes
1 answer
KRaft design architecture - Separation of Controllers and Brokers
We want to install Kafka Kraft cluster on 35 machines ( machines are DELL physical machines - type R760 And OS is Linux RHEL 8.6 version )
Since we are dealing with Kafka production cluster , we want to understand what is the best Practice and the…
jessica
- 2,426
- 24
- 66
0
votes
0 answers
KAFKA 3.5.1 create replica on controller
I have configuration 5 nodes, 4 broker+controller and 1 controller.
Create topic with replication factor 5, and it is created, and describe show that topic partition have 5 replicas.
/opt/kafka/latest/bin/kafka-topics.sh --create…
0
votes
0 answers
Kafka Kraft Connection to node -n could not be established. Broker may not be available (org.apache.kafka.clients.NetworkClient)
I'm trying to deploy kafka from kraft to 3 VM, according to the instructions https://kafka.apache.org/quickstart .
But when creating topic, the following error pops up and topic is not created
Here are my…
0
votes
0 answers
Has anyone got a working configuration for Kafka Kraft with SASL_SSL or even just SSL enabled?
I'm trying to get a Kafka cluster with Kraft off the ground. Plan to use it in production, and the Kafka documentation recommends not running a node in the "broker,controller" mode in production. So I'm splitting up the controllers and the brokers…
Aditya
- 41
- 1
- 5
0
votes
0 answers
Kafka error invalid recieve size with minus value
I'm running kafka kraft with three nodes, but the kafka show logs receive invalid size
org.apache.kafka.common.network.InvalidReceiveException: Invalid receive (size = -720899)
at…
Meikel Rizky Hartawan
- 29
- 1
- 1
- 7
0
votes
1 answer
Kafka without zookeeper + brokers failed after couple min
I am using Apache Kafka V3.5 in order to create 3 Kafka machines in the cluster ,
the goal is to have 3 broker machines without zookeeper
I am facing a problem after we installed and configured the kakas machines , and the problem is about that…
Judy
- 1,595
- 6
- 19
- 41
0
votes
1 answer
Kafka get stuck in adding replicas
I have testing cluster with 3 Kafka instances in KRaft mode. Each instance is Broker and Controller. I have inter communication secured with SSL certificate.
After cluster start, each instances know about each. When I create new topic with replicas,…
Adavan
- 63
- 2
- 6
0
votes
0 answers
Configuration for kafka +kraft high availability so on network split local nodes keep working
We need to set up a kafka cluster with L nodes (with L kraft controllers). There are N producers and N + M consumers and we are interested only in "live" data.
N consumers are classified as 'local' so they are intended to receive data from their…
0
votes
2 answers
Why do I need to specify all the brokers in Kafka with Kraft in a 3 node broker/controller cluster?
Deployed a 3 node kafka cluster in kubernetes using a statefulset using Kraft.
Within the cluster, the configuration is as follows (pseudo-code):
controller quorum: 0@kafka-0,1@kafka-1,2@kafka-2
listeners: …
StevenPG
- 311
- 4
- 16
0
votes
0 answers
Apache Kafka kraft controller dynamic configuration update
I will make this short, It is possible to update broker configurations dynamically with this command;
kafka-configs.sh
The entity-type can be topic, quota, brokers etc. but it cant be controller. That is not an option.
Since Kafka raft controllers…
musmil
- 53
- 6
0
votes
2 answers
Can I use Kafka in KRaft mode on AWS MSK?
I would like to use Kafka in KRaft mode instead of Zookeeper on AWS, since Kraft mode seems to be the future of Kafka. As I understand, basic solution from AWS for Kafka is AWS MSK. But in MSK documentation I see that they mention that they use…
rsnlpn
- 95
- 1
- 10
0
votes
1 answer
kafka kraft running in docker and windows wsl2
I'm trying to create a docker-compose with kafka kfrat but without success.
I'm using bitnami image instead of confuentinc.
Follow my docker compose:
version: "3.9"
services:
controller:
image: "bitnami/kafka:3.4.0"
hostname: "controller"
…
Rodrigo Batista
- 148
- 1
- 10
0
votes
1 answer
org.apache.kafka.raft.errors.NotLeaderException: Append failed because the replication is not the current leader
Introduction:
Previously, I saw a similar question (this link), but mine is different as we use Kafka KRaft instead of Kafka with Zookeeper.
Specification:
Kafka version: 3.3.1
Number of brokers: 8
Minimum replication factor of topics: 3
Problem…
Mostafa Ghadimi
- 5,883
- 8
- 64
- 102
0
votes
0 answers
kafka brokers cannot start because of EventHandler fail with create partition topic
i'm running with kafka 3.2.0 with kraft mode
when i restart the kafka brokers, the brokers keep fatal start up
it keep throwing the same error log
2023-01-05 04:36:00,267 - ERROR [EventHandler:Logging@76] - [BrokerMetadataPublisher id=1] Error…
onemin
- 11
- 2