Handle error globally and specific to channel while reading Aws Kinesis Streams in Spring Intehration

Question

I am listening to aws kinesis streams and enabled error handling globally and specific to channel as below.

Application Yml

server:
  port: 8090

eureka.client.enabled: false

spring:
  cloud:
    stream:
      bindings:        
        MyInboundChannel:
          group: myGroup
          destination: awsstream
          content-type: application/json
          errorChannelEnabled: true
          errors:
            destination: myGrouperrorChannel


cloud:
  aws:
    region:
      static: us-east-1
    credentials:
      accessKey: accessKey
      secretKey: secretKey
    kinesis:
      endpoint: endpoint

I have written the handler for global error channel as below

Error Handler

public class ErrorHandler {

  @ServiceActivator(inputChannel = "errorChannel")
  public void errorChannel(Throwable message) {
    log.error("error has been reported " + message);
  }

}

Configuration for my channel specific error channel

  @Bean(name = "myGrouperrorChannel")
  public MessageChannel myGrouperrorChannel() {
    return new DirectChannel();
  }

Message Listener and channel error listner

@EnableBinding(Binder.class)
public class MessageListener {

  @StreamListener("MyInboundChannel") 
  public void receiveMessage(String message) {
    System.out.println("MyInboundChannel" + message);
    //throw exception explicitly
    throw new RuntimeException("Iris error");
  }


    @StreamListener("myGrouperrorChannel")
  public void myGrouperrorChannel(Message message) {
    log.error("myGrouperrorChannel has been reported " + message);
  }

}

Binder Interface

public interface Binder {

  @Input("MyInboundChannel")
  SubscribableChannel itemMessage();

}

The error is neither handled by global errorChannel or my channel specific error channel

Am I missing any thing.

Exception Logs

Exception in thread "-kinesis-consumer-1" org.springframework.messaging.MessagingException: Exception thrown while invoking com.package.MessageListener  #receiveMessage[1 args]; nested exception is java.lang.RuntimeException: my error, failedMessage=GenericMessage [payload={hello
}, headers={aws_partitionKey=partitionKey-0, aws_shard=shardId-000000000000, aws_sequenceNumber=99579810409965332842672442827229165777883733894461652994, id=12d9f4a8-b5d2-2252-647f-b28bad087f5c, contentType=application/json, aws_stream=awsstream, timestamp=1519149049433}]
    at org.springframework.cloud.stream.binding.StreamListenerMessageHandler.handleRequestMessage(StreamListenerMessageHandler.java:63)
    at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:109)
    at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:127)
    at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:116)
    at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:148)
    at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:121)
    at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:89)
    at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:425)
    at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:375)
    at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:115)
    at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:45)
    at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:105)
    at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutput(AbstractMessageProducingHandler.java:360)
    at org.springframework.integration.handler.AbstractMessageProducingHandler.produceOutput(AbstractMessageProducingHandler.java:271)
    at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutputs(AbstractMessageProducingHandler.java:188)
    at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:115)
    at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:127)
    at org.springframework.integration.channel.FixedSubscriberChannel.send(FixedSubscriberChannel.java:70)
    at org.springframework.integration.channel.FixedSubscriberChannel.send(FixedSubscriberChannel.java:64)
    at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:115)
    at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:45)
    at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:105)
    at org.springframework.integration.endpoint.MessageProducerSupport.sendMessage(MessageProducerSupport.java:188)
    at org.springframework.integration.aws.inbound.kinesis.KinesisMessageDrivenChannelAdapter.access$5100(KinesisMessageDrivenChannelAdapter.java:82)
    at org.springframework.integration.aws.inbound.kinesis.KinesisMessageDrivenChannelAdapter$ShardConsumer.processRecords(KinesisMessageDrivenChannelAdapter.java:912)
    at org.springframework.integration.aws.inbound.kinesis.KinesisMessageDrivenChannelAdapter$ShardConsumer.access$3600(KinesisMessageDrivenChannelAdapter.java:688)
    at org.springframework.integration.aws.inbound.kinesis.KinesisMessageDrivenChannelAdapter$ShardConsumer$2.run(KinesisMessageDrivenChannelAdapter.java:822)
    at org.springframework.integration.aws.inbound.kinesis.KinesisMessageDrivenChannelAdapter$ConsumerInvoker.run(KinesisMessageDrivenChannelAdapter.java:1003)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: my error
    at com.package.MessageListener.receiveMessage(MessageListener.java:24)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:180)
    at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:112)
    at org.springframework.cloud.stream.binding.StreamListenerMessageHandler.handleRequestMessage(StreamListenerMessageHandler.java:55)
    ... 30 more

Can some one help

Update I have tried with below code but unable to catch error. I tried both Stream Listener and service activator. Can you please help me if I am doing any thing wrong.

Application Yml

    server.port: 8082

    spring:
      cloud:
        stream:
          bindings:
            output:
              destination: awsstream.myprocessor
              content-type: application/json
              producer:
                partitionKeyExpression: "1"
            input: 
              group: grp
              destination: awsstream.mysource
              content-type: application/json
              errorChannelEnabled: true
            input2:
              group: grp2
              destination: awsstream.mysink
              content-type: application/json

Class

        @EnableBinding(Processor.class)
        @SpringBootApplication
        public class MyApplication {

            public static void main(String[] args) {
                SpringApplication.run(MyApplication.class, args);
            }

            @StreamListener(Processor.INPUT)
            public void transform(String message) {
                if ("junk".equals(message)) {
                    System.out.println("processor"+message);
                }
                else {
                    throw new IllegalStateException("Invalid payload: " + message);
                }
            }

            @Bean(name = Processor.INPUT + "." + "grp" + ".errors")
            public SubscribableChannel consumerErrorChannel2() {
                return new PublishSubscribeChannel();
            }

            @ServiceActivator(inputChannel="errorChannel")
            public void processMessage(ErrorMessage message) {
                System.out.println("Errror has been reported errorMessage"+message);
            }

            @StreamListener("errorChannel")
            public void processMessagea(ErrorMessage message) {
                System.out.println("Errror has been reported errorMessage"+message);
            }

            @StreamListener("errorChannel")
            public void processMessagea(Message message) {
                System.out.println("Errror has been reported errorMessage"+message);
            }

            @ServiceActivator(inputChannel="errorChannel")
            public void processMessage2(Message message) {
                System.out.println("Errror has been reported message"+message);
            }

            @ServiceActivator(inputChannel=Processor.INPUT + "." + "grp" + ".errors")
            public void processMessage22(Message message) {
                System.out.println("Errror has been reported message"+message);
            }

            @ServiceActivator(inputChannel=Processor.INPUT + "." + "grp" + ".errors")
            public void processMessageg(ErrorMessage message) {
                System.out.println("Errror has been reported errorMessage"+message);
            }

            @StreamListener(Processor.INPUT + "." + "grp" + ".errors")
            public void processMessageaddd(ErrorMessage message) {
                System.out.println("Errror has been reported errorMessage"+message);
            }

            @StreamListener(Processor.INPUT + "." + "grp" + ".errors")
            public void processMessageaas(Message message) {
                System.out.println("Errror has been reported errorMessage"+message);
            }



        }

And error is not caught any where

Exception in thread "-kinesis-consumer-1" org.springframework.messaging.MessagingException: Exception thrown while invoking com.company.cloud.MyApplication#transform[1 args]; nested exception is java.lang.IllegalStateException: Invalid payload: aasa, failedMessage=GenericMessage [payload=aasa, headers={aws_partitionKey=partitionKey-0, aws_shard=shardId-000000000000, aws_sequenceNumber=49579810409608520919496497856488469248195805892912873474, id=2595d37f-82e1-b6bd-beed-3a9a3bb7243e, contentType=application/json, aws_stream=awsstream.mysource, timestamp=1520087287991}]
    at org.springframework.cloud.stream.binding.StreamListenerMessageHandler.handleRequestMessage(StreamListenerMessageHandler.java:63)
    at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:109)
    at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:127)
    at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:116)
    at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:148)
    at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:121)
    at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:89)
    at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:425)
    at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:375)
    at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:115)
    at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:45)
    at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:105)
    at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutput(AbstractMessageProducingHandler.java:360)
    at org.springframework.integration.handler.AbstractMessageProducingHandler.produceOutput(AbstractMessageProducingHandler.java:271)
    at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutputs(AbstractMessageProducingHandler.java:188)
    at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:115)
    at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:127)
    at org.springframework.integration.channel.FixedSubscriberChannel.send(FixedSubscriberChannel.java:70)
    at org.springframework.integration.channel.FixedSubscriberChannel.send(FixedSubscriberChannel.java:64)
    at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:115)
    at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:45)
    at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:105)
    at org.springframework.integration.endpoint.MessageProducerSupport.sendMessage(MessageProducerSupport.java:188)
    at org.springframework.integration.aws.inbound.kinesis.KinesisMessageDrivenChannelAdapter.access$5100(KinesisMessageDrivenChannelAdapter.java:82)
    at org.springframework.integration.aws.inbound.kinesis.KinesisMessageDrivenChannelAdapter$ShardConsumer.processRecords(KinesisMessageDrivenChannelAdapter.java:912)
    at org.springframework.integration.aws.inbound.kinesis.KinesisMessageDrivenChannelAdapter$ShardConsumer.access$3600(KinesisMessageDrivenChannelAdapter.java:688)
    at org.springframework.integration.aws.inbound.kinesis.KinesisMessageDrivenChannelAdapter$ShardConsumer$2.run(KinesisMessageDrivenChannelAdapter.java:822)
    at org.springframework.integration.aws.inbound.kinesis.KinesisMessageDrivenChannelAdapter$ConsumerInvoker.run(KinesisMessageDrivenChannelAdapter.java:1003)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Invalid payload: aasa
    at com.company.cloud.MyApplication.transform(MyApplication.java:39)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:180)
    at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:112)
    at org.springframework.cloud.stream.binding.StreamListenerMessageHandler.handleRequestMessage(StreamListenerMessageHandler.java:55)

May we have logs for the exception at least? I would like to see a call stack to determine the culprit if that one — Artem Bilan, Feb 20 '18 at 03:12
@ArtemBilan Thanks for looking into it. I intentionally threw RuntimeException from listener and added logs. Can you take a look — Patan, Feb 20 '18 at 18:02
I’ll talk to @GaryRussell tomorrow about this. Not sure what’s going on — Artem Bilan, Feb 22 '18 at 02:19
The `errorChannelEnabled` is a `producer` property. It is wrong on the consumer side — Artem Bilan, Feb 23 '18 at 16:06
@ArtemBilan Thank you. My bad I did not notice it. So, is there any way to catch all exceptions related to specific or group of consumers in one place? — Patan, Feb 23 '18 at 16:10
@ArtemBilan I have tried but still unable to catch error at consumer side. I have updated the question with code. Can you please check. It would be very great to get this working — Patan, Mar 03 '18 at 14:40
That’s too much custom code. The solution works for me, so now it is a time to share with us a simple project to let us to play and reproduce the problem in close to yours environment. Right it sounds like we use different versions — Artem Bilan, Mar 03 '18 at 17:36
@ArtemBilan. I have created the [POC](https://github.com/muzimil/spring-aws-poc). I have few restrictions. I needed to use spring 1.5.9 and cannot use any milestones or RC. Hence I have added the Kinesis binder with only few classes as required as libs. — Patan, Mar 04 '18 at 08:29
Wow! You don't use Kinesis Binder at all. And that one can be used only already with Spring Boot `2.0`. It is going to be a bit hard for you to achieve requirements if you don't use existing tools. — Artem Bilan, Mar 05 '18 at 15:46
@ArtemBilan. If I switch to spring boot 2.0.0.Release, do I still need to use Milestone version of kinesis binder right? — Patan, Mar 05 '18 at 18:27
Correct. And everything we've shown you here was based on that foundation. Would be even better to switch to the `BUILD-SNAPSHOT` there are a bunch of fixes done. I think we will release `M2` somewhere the next week together with the Spring Cloud Stream `2.0 GA` — Artem Bilan, Mar 05 '18 at 18:35

score 2 · Accepted Answer · answered Feb 23 '18 at 17:47

You know it works as designed. Only the problem that it's not so obvious.

I have just pushed an integration test to the Kinesis Binder project.

You can find it in the KinesisBinderProcessorTests.

But general idea is like this:

@RunWith(SpringRunner.class)
@SpringBootTest(
        webEnvironment = SpringBootTest.WebEnvironment.NONE,
        properties = "spring.cloud.stream.bindings.input.group = " + KinesisBinderProcessorTests.CONSUMER_GROUP)
@DirtiesContext
public class KinesisBinderProcessorTests {

    static final String CONSUMER_GROUP = "testGroup";

...
    @EnableBinding(Processor.class)
    public static class ProcessorConfiguration {

        @Transformer(inputChannel = Processor.INPUT, outputChannel = Processor.OUTPUT)
        public String transform(Message<String> message) {
            String payload = message.getPayload();
            if (!"junk".equals(payload)) {
                return payload.toUpperCase();
            }
            else {
                throw new IllegalStateException("Invalid payload: " + payload);
            }
        }

        @Bean(name = Processor.INPUT + "." + CONSUMER_GROUP + ".errors")
        public SubscribableChannel consumerErrorChannel() {
            return new PublishSubscribeChannel();
        }

Pay attention to the group property and how it is used to declare the consumerErrorChannel.

So, what you need instead of @StreamListener("myGrouperrorChannel") to use @StreamListener("MyInboundChannel.myGroup.errors"). To use the destination definition on the @StreamListener is wrong.

I also check in the test that errors from the consumer are sent to the global errorChannel. Not sure what's going on there for you...

You can add a `@ServiceActivator` (or any subscriber) on the global `errorChannel` (all inputs) or for a specific input `myInboundChannel.myGroup.errors` (unless the Kinesis binder customizes it. Each individual error channel is a pub/sub channel that is bridged to the global `errorChannel`. The payload is an `ErrorMessage` so a `@StreamListener` might not be appropriate. See the documentation [here](https://docs.spring.io/spring-cloud-stream/docs/Ditmars.SR2/reference/htmlsingle/index.html#binder-error-channels). — Gary Russell, Feb 23 '18 at 19:20
@GaryRussell I have tried but still unable to catch error at consumer side. I have updated the question with code. Can you please check — Patan, Mar 03 '18 at 14:39

Handle error globally and specific to channel while reading Aws Kinesis Streams in Spring Intehration

1 Answers1