I was wondering how does spring-xd deals with processors in a stream. What I really would like to know is if processors are blocking code, or they are related to how reactor (https://github.com/reactor/reactor/wiki/Processor) deals with processors.
If I need to execute expensive blocking operations (aka call an outside system), what is the best way of doing? I'd love to use reactor or any other reactive framework for that, but how to do so within XD pipeline architecture?
Regards
The term processor in a Spring XD stream has a specific meaning - it is basically a Spring Integration message flow from a channel named input to a channel named output. These channels, by convention, are what produce and consume the payload in an XD stream. For example, if a stream mystream is defined as someSource | someProcessor | someSink, the processor module may execute an expensive operation asynchronously but the stream still has to wait for a message on the processor's output channel, so you won't see improved throughput.
However there are some situations in which implementing a sink to run asynchronously would help. In this case the stream will not block when the message arrives on the sink's input channel. The async sink (kind of has a ring to it) could be attached to a tap on a stream:
mystream = someSource | ... | someSink
mytap = tap:stream:mystream > asyncSink
or a named queue (or topic):
mystream = someSource | ... | > queue:myQueue
queue:myQueue > asyncSink
or it could be the sink for the primary stream.
To implement an asynchronous sink requires configuring a Spring Integration endpoint, e.g., a ServiceActivator to call an external service, with a poller and a task executor within the sink module. The endpoint polls a Pollable channel (the input channel itself might be declared as a queue channel for example). See http://docs.spring.io/spring-integration/reference/html/messaging-endpoints-chapter.html for details.
