0

I am using Kafka 0.10.0.0, in a cluster with 3 nodes

I have auto topic creation enabled on brokers and on all brokers I have added properties for specify number of partition and replicas 

num.partitions=2  
default.replication.factor=2

Now when I am querying topic description via command line, I am getting unexpected results. Sometimes its showing 1 partition for this topic and sometimes 2 partitions for this topic.

bin/kafka-topics.sh --describe --zookeeper IP1:2181,IP2:2181,IP3:2181 --topic T.topic1_  
Topic:T.topic1  PartitionCount:1    ReplicationFactor:2 Configs:  
Topic: T.topic1 Partition: 0    Leader: 2   Replicas: 2,3   Isr: 2


bin/kafka-topics.sh --describe --zookeeper IP1:2181,IP2:2181,IP3:2181 --topic T.topic1
Topic:T.topic1  PartitionCount:2    ReplicationFactor:2 Configs:  
Topic: T.topic1 Partition: 0    Leader: 3   Replicas: 3,0   Isr: 0,3  
Topic: T.topic1 Partition: 1    Leader: 0   Replicas: 0,2   Isr: 2,0

Am I querying incorrectly since when its showing 1 partition its showing replicas 2 but ISR only 1

Also, the consumer group for this topic is always showing that consumer is listening on 2 partitions bin/kafka-consumer-groups.sh --new-consumer --bootstrap-server IP1:9092,IP2:9092,IP3:9092 --describe --group Group1 

GROUP                          TOPIC                          PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             OWNER
Group1    topic1         0          5               5               0               consumer-58_/10.20.1.20
Group1    topic1         1          3               3               0               consumer-58_/10.20.1.20

Also, specifying my broker configurations : -- zookeeper.properties

zookeeper.server=IP1:2181,IP2:2181,IP3:2181 (not sure need to give all ZK servers)

-- server.properties

num.partitions=2
default.replication.factor=2
zookeeper.connect=IP1:2181,IP2:2181,IP3:2181

Note : Zookeeper and Kafka are running on all IP1,IP2,IP3 servers

Matthias J. Sax
  • 59,682
  • 7
  • 117
  • 137
nikhil7610
  • 635
  • 1
  • 8
  • 13
  • There is no Kafka `2.11` (this is the Scala version of your binary). Latest Kafka version is `0.10.0.1`. – Matthias J. Sax Aug 20 '16 at 08:11
  • Thanks @MatthiasJ.Sax for correcting, I am using kafka version 0.10.0.0 – nikhil7610 Aug 20 '16 at 08:23
  • You should edit your question. Nobody wants to read all comments. :) – Matthias J. Sax Aug 20 '16 at 08:29
  • My first impression is that you are having problems with brokers or you are doing your first query before balancing is completed. – Luciano Afranllie Aug 22 '16 at 12:32
  • There is an extra _ after the topic name in your first command, I assume that this was not in the actual command executed and somehow stole into the post? Other than that I agree with Luciano, looks like an issue with brokers going down, perhaps check on your live brokers if it happens again: `zookeeper-shell IP1:2181 <<< "ls /brokers/ids"` – Sönke Liebau Feb 08 '17 at 10:18

0 Answers0