1

I have followed the below documentation and I have a producer and Consumer working perfectly fine with Kinesis Stream. I would like to understand how to handle the ERROR in Producer (Source) in case of any exception happens.

I have tried below approaches as per Spring Stream Error Handling Documentations:

@StreamListener("errorChannel")
@ServiceActivator(inputChannel = "errorChannel")

Both are not working. I am throwing a RuntimeException explicitly in the producer method and expecting that will be in the "errorChannel" but unable to receive the same.

Please help me to figure out this or share it with me approach if someone successfully did this.

Ferin Patel
  • 3,424
  • 2
  • 17
  • 49
user3792889
  • 79
  • 9

1 Answers1

0

Note: I have come across this exact problem few months ago. We were using Kafka in place of Kinesis. Since, you have tagged spring-cloud-stream-binder-kafka, I am giving the inputs on the same ground.

You can write a custom call back for your producer and this call back can tell you if the message has failed or successfully published. On failure, log the meta data for the message.

Below is a small code snippet than can explain better about the call back I was talking about.

Please note that this is for Kafka. Change the code accordingly for Kinesis.

public class ProduceToKafka {

  private ProducerRecord<String, String> message = null;

 // TracerBulletProducer class has producer properties
  private KafkaProducer<String, String> myProducer = TracerBulletProducer
      .createProducer();

  public void publishMessage(String string) {

    ProducerRecord<String, String> message = new ProducerRecord<>(
        "topicName", string);

    myProducer.send(message, new MyCallback(message.key(), message.value()));
  }

  class MyCallback implements Callback {

    private final String key;
    private final String value;

    public MyCallback(String key, String value) {
      this.key = key;
      this.value = value;
    }


    @Override
    public void onCompletion(RecordMetadata metadata, Exception exception) {
      if (exception == null) {
        log.info("--------> All good !!");
      } else {
        log.info("--------> not so good  !!");
        log.info(metadata.toString());
        log.info("" + metadata.serializedValueSize());
        log.info(exception.getMessage());

      }
    }
  }

}

I wrote a medium article of handling the producer failures at different levels of Spring abstractions. This can help you. Check it out : https://medium.com/@akhil.ghatiki/kafka-producer-failure-handling-at-multiple-levels-of-spring-abstractions-e530edb02a6c

Akhil Ghatiki
  • 1,140
  • 12
  • 29
  • Thanks for providing this Akhil. However, I am using the Spring Data Flow Source Binding and it has only 2 types of parameterized send() method which will not accept the Callback. this.source.output().send(message); – user3792889 Aug 23 '20 at 14:48
  • Welcome !! in that case try auto wiring the producer listener and implement your custom logic in the new implementation. Have a look at the "Implementing in Spring cloud data stream" section in the link I shared in above answer, it might give you sine insights. – Akhil Ghatiki Aug 23 '20 at 16:01
  • Thank you, Akhil. Unfortunately, the Kinesis Binder is not having the ProducerListener to Autowire. – user3792889 Aug 31 '20 at 08:05