0

I have a topic first_topic with 3 partitions.

enter image description here

Here is my code, I send 55 messages to a consumer ( which is running in cmd ), the code below shows the partition to which the message was sent. Every time I launch the code all the messages go to one partition only ( that is picked randomly ), it may be partition 0, 1 or 2. Why doesn't round-robin work here? I do not specify the key, so I hope it should.

   Logger logger = LoggerFactory.getLogger(Producer.class);
    Properties properties = new Properties();
    properties.setProperty(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "127.0.0.1:9092");
    properties.setProperty(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
    properties.setProperty(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());


    KafkaProducer<String, String> producer = new KafkaProducer(properties);


    for (int i = 0; i < 55; i++) {


    producer.send(new ProducerRecord<>("first_topic", "Hello world partitionCheck " + i), (recordMetadata, e) -> {
        // executes every time a record is sent or Exception is thrown
        if (e == null) {
            // record was successfully sent
            logger.info("metaData: " +
                    "topic " + recordMetadata.topic() +
                    " Offset " + recordMetadata.offset() +
                    " TimeStamp " + recordMetadata.timestamp() +
                    " Partition " + recordMetadata.partition());
        } else {
            logger.error(e.toString());
        }
    });
    }

    producer.flush();
    producer.close();
Klaus Köhler
  • 143
  • 4
  • Does this answer your question? [Kafka RoundRobin partitioner not distributing messages to all the partitions](https://stackoverflow.com/questions/65363484/kafka-roundrobin-partitioner-not-distributing-messages-to-all-the-partitions) – Chin Huang Aug 08 '22 at 16:38

1 Answers1

1

As per theory, if you are not specifying any custom partition it will use the default partitioner as per the below rule

  • If a partition is specified in the record, use it that partition to publish.
  • If no partition is specified but a key is present choose a partition based on a hash of the key
  • If no partition or key is present choose a partition in a round-robin fashion

Can you confirm, how are you checking this. "Every time I launch the code all the messages go to one partition only ( that is picked randomly ), it may be partition 0, 1 or 2. "

ChristDist
  • 572
  • 2
  • 8
  • Thank you for the answer, you can see in my code " Partition " + recordMetadata.partition()); I get this data by passing new Callback() as a second argument in send method. But I can actually verify it using UI for Kafka. – Klaus Köhler Aug 08 '22 at 11:22