Kafka: This server is not the leader for that topic-partition

Question

Possible duplicate of Kafka - This server is not the leader for that topic-partition but there is no accepted answer nor a clear solution.

I have a simple java program to produce the message to Kafka:

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("acks", "all");
props.put("retries", 1);
props.put("batch.size", 16384);
props.put("linger.ms", 100);
props.put("buffer.memory", 33554432);
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "com.company.project.KafkaValueSerializer");
MyMessage message = new MyMessage();
Producer<String, MyMessage> producer = new KafkaProducer<>(props);
Future<RecordMetadata> future =
    producer.send(new ProducerRecord<String, MyMessage>("My_Topic", "", message));

I am getting the following exception:

Exception in thread "main" java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.NotLeaderForPartitionException: This server is not the leader for that topic-partition.
    at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.valueOrError(FutureRecordMetadata.java:94)
    at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.get(FutureRecordMetadata.java:64)
    at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.get(FutureRecordMetadata.java:29)
    at 
Caused by: org.apache.kafka.common.errors.NotLeaderForPartitionException: This server is not the leader for that topic-partition.

When I tried with kafka-console-producer, I am getting the following error:

D:\kafka_2.11-0.10.2.0\bin\windows>kafka-console-producer.bat --broker-list localhost:9092  --topic My_Topic
hello
[2018-10-25 17:05:27,225] WARN Got error produce response with correlation id 3 on topic-partition My_Topic-0, retrying (2 attempts left). Error: NOT_LEADER_FOR_PARTITION (org.apache.kafka.clients.producer.internals.Sender)
[2018-10-25 17:05:27,335] WARN Got error produce response with correlation id 5 on topic-partition My_Topic-0, retrying (1 attempts left). Error: NOT_LEADER_FOR_PARTITION (org.apache.kafka.clients.producer.internals.Sender)
[2018-10-25 17:05:27,444] WARN Got error produce response with correlation id 7 on topic-partition My_Topic-0, retrying (0 attempts left). Error: NOT_LEADER_FOR_PARTITION (org.apache.kafka.clients.producer.internals.Sender)
[2018-10-25 17:05:27,544] ERROR Error when sending message to topic My_Topic with key: null, value: 5 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.NotLeaderForPartitionException: This server is not the leader for that topic-partition.

When I describe my topic, I have the following information:

Topic:My_Topic        PartitionCount:1        ReplicationFactor:1     Configs:
        Topic: My_Topic       Partition: 0    Leader: 0 Replicas: 0     Isr: 0

I tried creating a new topic and producing messages as mentioned in Quick start guide then the above steps work well.

What correction should I make in My_Topic or producer configuration so that I can publish messages successfully in this topic?

Can you try to increase number of retries and let me know the outcome? For example` props.put("retries", 10);` — Giorgos Myrianthous, Oct 25 '18 at 12:39
@GiorgosMyrianthous: nothing changed. it remains with the same exception and stacktrace. — Mohammad Faisal, Oct 25 '18 at 12:51

score 0 · Answer 1 · answered Dec 11 '18 at 17:22

Had it been the case of "console client working, but Java program not working", then the solution of 'changing retry limit' may have helped.

As both the Java program and the built-in command-line producer are not able to connect to Kafka, I suspect that the problem could be due to stale configuration.

(Example: topic deleted and created again with the same name, but with different partition count).

Deleting the old log files of zookeeper and Kafka and creating the topic again, or creating a topic with a different name will resolve the issue.

Kafka: This server is not the leader for that topic-partition

1 Answers1