2

I need to define multiple streams from multiple sources and then join them into one. After reading the whole documentation of Spring XD, I did not find any predefined module that can do a join (based on sliding windows for example). Is there any solution to do that ? thanks.

user3906228
  • 21
  • 1

1 Answers1

5

It is possible to do this with named channels. For example you can create several streams that output to a queue, and a "join" stream which uses the queue as a source:

xd:>stream create s1 --definition "http --port=9000 > queue:join" --deploy
xd:>stream create s2 --definition "http --port=9001 > queue:join" --deploy
xd:>stream create join --definition "queue:join > file" --deploy

The above will write content posted to either http://somehost.example.com:9000 or port http://somehost.example.com:9001 to a file.

dturanski
  • 1,723
  • 1
  • 13
  • 8
  • 1
    Thanks for your response but this is like a merge not a join. A join means correlating tuples from each input stream in order to create new tuples to the output stream. In streaming, we commonly use sliding temporal windows to join tuples from multiple input streams and correlating them. It seems like sliding windows are not covered in Spring XD? – user3906228 Aug 05 '14 at 08:22
  • For "joining" purposes, you can have a look at the aggregator module (a processor) that retains messages and correlates them using some pluggable logic. You may be interested in the Spring Integration documentation of this (http://docs.spring.io/spring-integration/docs/4.1.0.BUILD-SNAPSHOT/reference/html/messaging-routing-chapter.html#aggregator) – ebottard Aug 05 '14 at 09:35
  • This doesn't work as a join or a merge infact. Your file merely will have records posted on both sources. The aggregator module could be configured to meet your need probably. – Arun Jose Aug 12 '14 at 09:40