0

I am using a Gateway and inside the gateway I am looping over to send the message to 2 queues via sqs-outbound adapter.

I want to achieve something like this:

jsonArray.forEach(data -> {
    Future<AggregatedAckorErrorFrom2Queues> result = dataPublishGateway.publishToService(data);
    futures.add(result);
});
futures.forEach(f -> {                      
    System.out.println("Future Result -> "+ f.get())
});

The Gateway and SQS Adapter Config

<!-- Gateway to Publish to SQS -->
<task:executor id="dataExecutor" pool-size="5"/>
<int:publish-subscribe-channel id="dataChannel" task-executor="dataExecutor" />
<int:gateway service-interface="com.integration.gateway.PublishGateway" id="dataPublishGateway">
    <int:method name="publishToDataService" payload-expression="#args[0]" request-channel="dataChannel" />
</int:gateway>

<!-- Send to 2 Queues -->

    <int:channel id="successChannel"/>
    <int-aws:sqs-outbound-channel-adapter sqs="amazonSQS"
                                          queue="queue-1"
                                          channel="dataChannel"
                                          success-channel="successChannel"
                                          failure-channel="errorChannel"/>
    <int-aws:sqs-outbound-channel-adapter sqs="amazonSQS"
                                          queue="queue-2"
                                          channel="dataChannel"
                                          success-channel="successChannel"
                                          failure-channel="errorChannel"/>

I am looking at apply-sequence on the dataChannel and an aggregator, but the aggregator must be able to process both ack and error.

Question : How can I return an aggregated response(ack + error) from 2 queues back to the gateway?

Nikhil
  • 345
  • 2
  • 13

1 Answers1

1

You are on the right direction with those success-channel and failure-channel. The aggregator can subscribe to that successChannel and do correlation and release using default strategies. With an failure-channel you probably need a custom error channel, not that one global errorChannel. The ErrorMessage sent to that channel from the SqsMessageHandler has a payload like this AwsRequestFailureException, where its failedMessage really contains a request message with the mentioned correlation details. So, you need to add some transformation step to extract that info from the exception before proceeding to the aggregator.

Artem Bilan
  • 113,505
  • 11
  • 91
  • 118
  • How can I return the result to the gateway from this call say Thread 1 --> `Future result = dataPublishGateway.publishToService(data)` . Since the `dataPublishGateway` publishes to 2 SQS queues over a pub/sub channel (`dataChannel`), how can the calling thread (Thread 1 above) get results from the callbacks of these 2 queues? – Nikhil Sep 03 '20 at 17:55
  • The aggregator will do that automatically based on the `replyChannel` header from one of the message you are going to aggregated. And the waiting thread in the gateway will do correlation for that reply. – Artem Bilan Sep 03 '20 at 17:56
  • The thread does not wait since the subscriber to the dataChannel is a executor based subscriber. Anything I am missing? – Nikhil Sep 03 '20 at 17:58
  • You initiate the call from the `dataPublishGateway`, so there is indeed a request-reply behavior. Even if it is `Future` result, it still going to be fulfilled by the reply message from that `replyChannel` header. – Artem Bilan Sep 03 '20 at 18:00
  • Well, need to see what you are doing there within the `` – Artem Bilan Sep 03 '20 at 19:24
  • You are right, I was filtering the `replyChannel` in the chain via `header-filter`, my mistake, sorry! Everything is working just that I see this warning : `Message header with name 'replyChannel' and type 'org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel' cannot be sent as message attribute because it is not supported by SQS.` I will now focus on the aggregator solution as suggested, thank you very much for all your help ! – Nikhil Sep 03 '20 at 19:51
  • Well, since SQS cannot send that header it is filtered over there before prdocuing an SQS message. But that's just a warn which does not affect your application. I can't suggest you to try to filter that header somehow before sending to SQS, because we definitely need it after producing. We can convert it to string though, but do we really need it to go to SQS queue on AWS at all?.. – Artem Bilan Sep 03 '20 at 19:53
  • Well, there is an `SqsHeaderMapper` which you can configure to for the `setOutboundHeaderNames()` to avoid that header to pass: `!replyChannel`, for example. – Artem Bilan Sep 03 '20 at 19:55