0

We have a use case where messages are consumed from RabbitMQ, transform it and push the record to the Kinesis stream. Each message in RabbitMQ corresponds to a record in the Kinesis stream. AmazonKinesisAsync supports PutRecordsRequestEntry where multiple records can be grouped to make a single HTTP call.

My question is there any out-of-box component in Spring Integration that helps to group the messages from rabbitmq into a batch based on the configurable amount of records size before pushing to Kinesis?

LBPS
  • 83
  • 9

1 Answers1

1

See an Aggregator EI pattern and its implementation in Spring Integration: https://docs.spring.io/spring-integration/docs/current/reference/html/message-routing.html#aggregator. So, before pushing the message to the KinesisMessageHandler, you have an AggregatingMessageHandler for batching those message from RabbitMQ by some configured size and emitting a PutRecordsRequest container object.

Artem Bilan
  • 113,505
  • 11
  • 91
  • 118