I'm making project using springboot 2, kafk 2.2.0, spring-kafka 2.2.5
I maked kafka exactly once environment and message producing and consuming was well.
BUT kafka-consumer-groups.sh saied like this.
TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG
test_topic 0 23 24 1
test_topic 1 25 26 1
test_topic 2 21 22 1
I just send only one message to kafka, but LOG-END-OFFSET doubled up and 1 lag remain always. (In my java app, producing and consuming works as intended)
I don't know why LOG-END-OFFSET doubled up.
If removing exactly once config, there is no problem in LOG-END-OFFSET and CURRENT-OFFSET count.
It is my kafkaTemplate setup codes.
@Bean
@Primary
public ProducerFactory<String, Object> producerFactory() {
Map<String, Object> producerProperties = new HashMap<>();
producerProperties.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092";
producerProperties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
producerProperties.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, JsonSerializer.class);
// exactly once producer setup
producerProperties.put(ProducerConfig.ENABLE_IDEMPOTENCE_CONFIG, "true");
DefaultKafkaProducerFactory factory = new DefaultKafkaProducerFactory<>(producerProperties, new StringSerializer(), new JsonSerializer<>(KafkaStaticOptions.OBJECT_MAPPER));
factory.setTransactionIdPrefix("my.transaction.");
return factory;
}
@Bean
@Primary
public KafkaTransactionManager<String, Object> kafkaTransactionManager(
ProducerFactory<String, Object> producerFactory) {
return new KafkaTransactionManager<>(producerFactory);
}
@Bean
@Primary
public KafkaTemplate<String, Object> kafkaTemplate(ProducerFactory<String, Object> producerFactory) {
return new KafkaTemplate<>(producerFactory);
}
My producer code.
kafkaTemplate.executeInTransaction(kt -> kt.send("test_topic", "test data hahaha"));
I checked when LOG-END-OFFSET doubled up, and it is produce transaction commit timing.
What did I do wrong configuration?
