2

i'm facing the duplication data in kafka so i want to use idempotent in kafka-python but unfortunately its not the part of it, how i can stop the duplication data in same partition with out idempotent or there is any way to import it or customize the current library? Thanks in advance.

farhan pirzada
  • 187
  • 8

1 Answers1

2

You'd better use confluent_kafka, since kafka-python does not provide this feature. Have a look at this link

producer = Producer({'bootstrap.servers': ‘localhost:9092’,
          'enable.idempotence': True})

This feature was made available with the 1.0.0 release of librdkafka (a library written in C).

Peter
  • 399
  • 2
  • 6
  • 23