2

Is it possible to implement a complext parallel processors flow in spring cloud data flow using the preset modules?

For example:

Processor 1, 2, 3 are all preset modules (httpclient etc). Processor 1 and 3 will get the same message from source at the same time, then processor 1 pipes the output to processor 2. Upon both processor 2 and 3 complete, they pipe the output to processor 4 at the same time. The output from processor 2 is irrelevant, it is fine that processor 4 only gets the output from 3. However, processor 3 should only send messages to 4 when processor 2 is complete.

enter image description here

Is this something that can be accomplished using Spring Cloud Data Flow?

MFerguson
  • 1,739
  • 9
  • 17
  • 30
user3908406
  • 1,416
  • 1
  • 18
  • 32

1 Answers1

2

Using taps and named destinations you can get most of the way there.

Flow

The DSL looks like:

main-stream=source: http | processor1: script | processor2: script > :destination
second-stream=:main-stream.source > processor3: script > :destination
sink-stream=:destination > sink: log

Synchronizing between processor2 and processor3 sounds like an anti-pattern. It sounds more like you need processor3 to receive from processor2, rather than work in parallel (it just also needs an original copy of the message).

fiidim
  • 157
  • 8