Is there a Dataflow block like TransformBlock that allows items to be propagated out of sequence?

Question

I'm relatively new to asynchronous processing and experimenting with TPL Dataflow. My scenario: I have a block that is continuously supplied with input, asynchronously executes a function on the input, and returns results. (The results are then passed to another block that saves to the database.) The function may complete within milliseconds or it could take as long as 10 seconds to return. The block is set up to process with "Unbounded" parallelism. The ordering of the results has no importance.

I started out using a TransformBlock but (because it maintains the item sequence) a single slow item would cause many faster items to pile up behind it. In this specific situation it is not only acceptable for the results to propagate out of sequence but highly desirable. This allows the results to flow at a regular rate, instead of in giant waves.

I've searched several times, looking for an implementation of a Dataflow block that propagates items as soon as they are complete, but I've yet to find anything that matches this description. I gave up and created my own NonSequentialBlock by "duct-taping" together an ActionBlock and a BufferBlock. It seems to do the job, but I'm concerned that (in my lack of experience) I've done something wrong and it will bite me in the end. Is there an existing implementation of this pattern available?

Related: [How can I specify an unordered Execution Block using the TPL Dataflow Library?](https://stackoverflow.com/questions/22893908/how-can-i-specify-an-unordered-execution-block-using-the-tpl-dataflow-library) — Theodor Zoulias, Aug 31 '20 at 19:31

score 3 · Accepted Answer · answered Dec 11 '14 at 09:45

3

No, you haven't done anything wrong. It's typical to create your own blocks by combining the basic blocks, then calling Encapsulate to ... encapsulate them and present a single IPropagator block.

answered Dec 11 '14 at 09:45

Panagiotis Kanavos

120,703
13
188
236

That should work perfectly. I had no idea that the `Encapsulate` method existed. Thanks! – Michael Richardson Dec 11 '14 at 15:47

score 1 · Answer 2 · answered Aug 31 '20 at 19:28

A new option EnsureOrdered has been added to the DataflowBlockOptions class, that determines whether a dataflow block will output the processed results in the order of the input messages. This property is true by default. Setting this property to false makes the block to propagate the messages as soon as they are processed, ignoring their original order.

This option was introduced in 2016.

Is there a Dataflow block like TransformBlock that allows items to be propagated out of sequence?

2 Answers2