1

So, I created a couple of short flows, using SQS as the backing store between processing segments. The basic flow is:

RestController -> SQS Queue1 OCA
  SQS Queue1 MDCA -> Service-Adapter -> SQS Queue2 OCA
  SQS Queue2 MDCA -> Service Adapter

However, I ran into a couple of problems... The "SQS Queue1 MDCA" reads messages off the queue with AWS specific Message Headers that end up getting to the outbound adapter writing to Queue2. These message headers specify things like the AWS_queue name, aws message id, etc, which cause the message to be re-delivered to queue1.

Once I filtered these headers off (I used a simple custom transformer to test this), the attributes of the outbound-channel-adapter then work as expected and the QueueMessagingTemplate delivers the message to the proper queue.

So, my question for the spring-int and spring-aws experts is, "Where do you think the proper place to filter out any pre-existing SQS headers is so that they don't get picked up by any downstream SQS processing?" Seems to me like you'd want to do it in the SQSHandler, since they might be relevant in any base spring-aws-messaging calls.

As a side note, with a message being created from an SQS ICA, and an object-to-json transformer in the flow, it also caused 12 message headers to be created which is 2 more than is allowed by SQS (causing the message delivery to fail).

Sample message with headers slightly modified to protect the innocent queues... As you can see the aws_queue, destination, etc are still in the message from being read from "wrench_workflow-mjg" so that when trying to deliver to the next queue, those headers overwrote the configuration in the spring-int xml configuration and never delivered to the next queue. The message just kept cycling through SQS queue "wrench_workflow-mjg" (after fixing the more than 10 attributes problem).

GenericMessage [payload={"eventId":"event-1","eventStartDateTime":1476127827.201000000}, 
headers={aws_messageId=db9b6cc0-f133-4182-b79c-4d5d9717a3a9, ApproximateReceiveCount=1, 
SentTimestamp=1476127827803, id=0b662b72-f149-a970-5e63-64a1b28290fb, 
SenderId=AIDAJOYV7TECZCZCOK22C,
aws_receiptHandle=AQEBdaToWo9utbjBwADeSuChX/KrY3l8eoSBMZ0RMmbI8ewjJQw6tV74RwSYxWPKZBSzgJhCmfJ8AUX+
reOy2yGrmefULU7NS8nqYTdMW6BB4Ug2+mHIY+8Tze+2ws15FB5t96q3iJO8+tP5pl/xuo+CiTDv+
L1rlYkVODD0Yv1zosGKx48IhGOXdL8nJ4Im8bujcUNLJy/vEYD8Qcfsi6NHOF3Qn0A4Xw+Nva0wp86zfq,
aws_queue=wrench_workflow-mjg,
lookupDestination=wrench_workflow-mjg, 
ApproximateFirstReceiveTimestamp=1476127827886, timestamp=1476127836254}

I can probably pull together an example if necessary, but I'm hoping you understand what I'm getting at without.

MikeG
  • 23
  • 3

1 Answers1

0

We should probably add header-mapping facilities to the adapters to enable selective mapping of headers like we have for other technologies in Spring Integration.

I have opened a GitHub Issue.

In the meantime, you could put an <int:header-filter.../> (or HeaderFilter transformer) immediately upstream of the outbound adapter to remove the unwanted headers.

You could also add a custom Request Handler Advice to the outbound adapter.

Gary Russell
  • 166,535
  • 14
  • 146
  • 179
  • just off the top of your head, where is an example where the above header mapping is implemented? – MikeG Oct 12 '16 at 18:37
  • Immediately before the outbound adapter; if you can tell me whether you are using xml or @Bean configuration, I can edit my answer with config. – Gary Russell Oct 12 '16 at 19:16
  • Thanks, but not necessary. We're pretty heavy users of spring-int and we have already implemented your suggested workaround. If we can get this project out and I can get some time, I'll take a stab at the GitHub issue too. – MikeG Oct 14 '16 at 02:11
  • (Oh, and we tend to use XML for flow definitions - ironically, it seems easier to see the flows using that than the longer java config. Although the java DSL looks pretty good for simple flows. – MikeG Oct 14 '16 at 02:13
  • My first reply was asking in relation to the github issue - I should have asked there. Thanks again! – MikeG Oct 14 '16 at 02:18