enable.idempotence=true
librdkafka version is 1.6.0
Our external Kafka readers expect messages without any gaps or duplicates
When producing with librdkafka C++ API below 3 type of errors can be detected,
- From event callback, void RdKafka::EventCb::event_cb(RdKafka::Event& event), with event.fatal() == true.
- From Delivery callback, void RdKafka::DeliveryReportCb::dr_cb(RdKafka::Message& message), with "message.err() != RdKafka::ERR_NO_ERROR" OR "message.status() != RdKafka::Message::MSG_STATUS_PERSISTED"
- Return value of RdKafka::Producer::produce, is not "RdKafka::ERR_NO_ERROR" nor RdKafka::ERR__QUEUE_FULL
When any of these errors occur what is the recommended error handling procedure ?
Terminate ?
OR
Is it correct to [All the steps from 1 to 4 are part of the suggested solution]
- Destroy existing RdKafka::Producer object.
- Re-create RdKafka::Producer object.
- Using a consumer, read the last message that was written in the topic partition.
- Produce starting from next message from above #4 last written msg
OR
Any other ?