1

I am using AWS as a Cloud provider. I have a Microservice that is in Frankfurt Region and will publish events to a Kinesis Data stream in the same region using Spring Cloud Stream (SCDF) Kinesis Adapter. I have multiple Microservices in different regions (Oregon, Ohio, Singapore, Mumbai etc) which is consuming events from respective Kinesis Streams in the respective regions using Spring Cloud Stream (SCDF) Kinesis Adapter. Now I have to route the Events which are there in the Frankfurt Kinesis to different Data Streams in different regions (only related to respective Kinesis).

  1. Can I do this using any of the Spring provided functionality? Can I use Spring Cloud Stream or SCDF to do cross-region routing? if yes, please point to some examples.
  2. If #1 is not possible what are the best ways to do this?
  3. I read about AWS EventBridge, is it a correct choice for the above use case?
user3792889
  • 79
  • 9

1 Answers1

0

Spring Cloud Stream Binder for AWS Kinesis is fully based on a standard AWS Client or KCL. Both of them are require particular region to be configured staticaly or resolved from the EC2 environment. So, to be able to consume from one region and relay stream records to another, you have to code some "replicator" stream application.

Luckily Spring Cloud Stream application can be configured for several binders. Right, in our case both of them are going to be the same Kinesis binder, but we are going to configure them for different credentials and different regions.

See Spring Cloud Stream docs for multi-binder configuration: https://docs.spring.io/spring-cloud-stream/docs/3.1.2/reference/html/spring-cloud-stream.html#multiple-binders.

Probably the code of your Stream Application could be just a plain identity function:

    @Bean
    public Function<byte[], byte[]> kinesisStreamRelay() {
        return Function.identity(); 
    }

And you bind it for an in destination from one Kinesis binder and out destination in the other.

Also see other ways to do that in this article: https://engineering.opsgenie.com/cross-region-replication-of-kinesis-streams-4a62f3bb269d

See Spring Cloud Function support for AWS Lambda: https://docs.spring.io/spring-cloud-function/docs/3.1.1/reference/html/aws.html. Spring Cloud Stream does not provides binder implementation for AWS Lambda.

Artem Bilan
  • 113,505
  • 11
  • 91
  • 118
  • Thank you Artem for the suggestions and tips. Can you please share an example for the above-said identity function or reference. Here the source (in binder) is single and the destination (out binder) is multiple so using a Spring multi-binder approach may not work. Do you have any insight into using AWS EventBridge in place of a Sping Kinesis solution? – user3792889 May 23 '21 at 05:22