0

I would like develop a consumer library which would read kinesis stream names from spring properties file and create/activate associated consumers.

In all the examples I have seen (like https://github.com/spring-cloud/spring-cloud-stream-samples/blob/master/kinesis-samples/kinesis-produce-consume/src/main/java/demo/stream/OrderStreamConfiguration.java), consumers are annotation-based and they should be manually defined.

I want to know which class from spring-integration/spring-aws-kinesis library I could use to generically create consumers.

Thank you

sansari
  • 558
  • 1
  • 7
  • 17

1 Answers1

0

For dynamic and runtime purpose we suggest to use Spring Integration Java DSL and its IntegrationFlowContext: https://docs.spring.io/spring-integration/docs/5.0.5.RELEASE/reference/html/java-dsl.html#java-dsl-runtime-flows.

So, what you mean could be achieved with an iterator over those properties and an 

KinesisMessageDrivenChannelAdapter kinesisMessageDrivenChannelAdapter = ...;
IntegrationFlows.from(kinesisMessageDrivenChannelAdapter)... ;

combination.

Artem Bilan
  • 113,505
  • 11
  • 91
  • 118
  • So I need to create a channel adapter per stream? thanks! – sansari Jun 11 '18 at 19:56
  • Well, Actually, one `KinesisMessageDrivenChannelAdapter` can handle several streams. See its ctors. So, if you want, you really need only one adapter. Maybe then a `@Bean` for the `KinesisMessageDrivenChannelAdapter` would be enough for you? – Artem Bilan Jun 11 '18 at 19:59
  • oh, right! I can pass a list of streams to constructor. Then yeah, one adapter should be enough. Thanks for swift answer. – sansari Jun 11 '18 at 20:01
  • This did not actually answer the question. Java DSL is not acceptable for all use cases and the lack of namespace support, or more aptly, the incomplete namespace support for spring-integration-aws has exempted it from more than one project I have worked on and now as of today it is exempting it from a 3rd project. How would you go about dynamically loading an integration flow over HTTP that is programmed in Java DSL? Given my requirements this incomplete implementation of namespace support makes spring-integration-aws unusable. – user578888 Mar 17 '23 at 20:08
  • Then design of your project is wrong. I would call it an abuse of configuration purpose. The real Spring Integration application is not just a flow of endpoints: there are many other beans endpoints and channels are depend on. And I would say that many of them are these days without an XML configuration support. So, you need to revise you project design to some other way, then just complain that an XML configuration for this Spring Integration AWS extension would help you somehow. The new upcoming Spring Cloud AWS and new Spring Integration AWS will be fully without an XML configuration. – Artem Bilan Mar 20 '23 at 13:10