1

I have made a custom Partitioner class that extends default Partitioner.

Problem : I want to add this custom Partitioner in KafkaSender.send method()

KafkaSender.send method() code :

sender.send(Flux.just(SenderRecord.create(new ProducerRecord<>(topic, partition, key, record, recordHeaders), 1))))

The partitioner here is an integer

Custom Partitioner Code:

public class CustomPartitioner extends DefaultPartitioner {
private final static String CHAR_FORMAT = "UTF-8";
@Override
public int partition(String topic, Object key, byte[] keyBytes, Object value, byte[] valueBytes, Cluster cluster) {
    // my logic 
    try {
        return super.partition(topic, key, iocKey.toString().getBytes(CHAR_FORMAT), value, valueBytes, cluster);
    } catch (UnsupportedEncodingException e) {
       //error message
    }
}

}

Note : I tried to hard code it using this below code

     Properties properties = new Properties();
     properties.put("partitioner.class", "CustomPartitioner ");

How can we force KafkaSender.send method() to use our custom partitioner?

rushabh
  • 59
  • 3

1 Answers1

1

You have to pass the properties map to KafkaTemplate bean as part of your producer configuration.

@Bean
public KafkaTemplate<String, String> kafkaTemplate() {
    Map<String, Object> configProps = new HashMap<>();
    configProps.put("partitioner.class", "<packagename>.CustomPartitioner");
    return new KafkaTemplate<>(configProps );
}
Steephen
  • 14,645
  • 7
  • 40
  • 47
  • But how the KafkaSender.send method() will pick up this property? Also what should we pass here sender.send(Flux.just(SenderRecord.create(new ProducerRecord<>(topic, partition, key, record, recordHeaders), 1)))) – rushabh Nov 30 '20 at 08:05
  • 1
    KafkaTemplate.send() is the API to send message to Kafka using Spring-Kafka. So it will wire it internally. – Steephen Nov 30 '20 at 08:08
  • Okay so instead of using KafkaSender.send() , I should use KafkaTemplate.send() ? – rushabh Nov 30 '20 at 08:13
  • `KafkaSender` is from `reactor-kafka`; you need to set the properties in the `DefaultKafkaSender`'s `ProducerFactory`. – Gary Russell Nov 30 '20 at 16:32
  • So using KafkaTemplate can i set the property of DefaultKafkaSender's ProducerFactory – rushabh Dec 01 '20 at 05:31