TPL Dataflow: Persist previous data

Question

I have been using TPL dataflow for an image processing pipeline using the producer/consumer pattern.

I'm trying to work out the best approach to allow for algorithms that require either the previous frame or a persistent object.

An example of one of these processes is background subtraction where the foreground mask is found by comparing the current frame to a dynamic background model.

One idea of how to achieve this is to create a cycle in the pipeline:

Is there a standard way of approaching this sort of pipeline?

Are there any issues related to the ordering of data or the asynchronous operation?

Why not just keep a data member? – i3arnon Aug 12 '14 at 09:09 — i3arnon, Aug 12 '14 at 09:09

score 3 · Accepted Answer · answered Aug 12 '14 at 11:41

You don't need to complicate things by using a cycle in the pipeline, like in your image, all you need is to keep the persistent data in a variable that persists between calls to the processing function.

If you're using a lambda, that variable can be a local variable outisde of the lambda:

IPropagatorBlock<InputImage, OutputImage> CreateProcessingBlock()
{
    InputImage previousImage = null;

    return new TransformBlock<InputImage, OutputImage>(
        inputImage =>
        {
            var result = Process(inputImage, previousImage);
            previousImage = inputImage;
            return result;
        })
}

If you're using an instance method on some object, that variable can be an instance field on that object:

class Processor
{
    InputImage previousImage;

    public OutputImage Process(InputImage inputImage)
    {
        var result = Process(inputImage, previousImage);
        previousImage = inputImage;
        return result;
    }
}

…

new TransformBlock<InputImage, OutputImage>(new Processor().Process)

In the second example say I want to use an use instance method `myProcessor.Reset()`. Is there any issue passing `myProcessor.Process` to the block whilst also calling reset methods on the same object from outside the block? — John Redmond, Aug 14 '14 at 08:42
@JohnRedmond Yes, if those two methods access the same members, then you need to make sure they do that in a thread-safe manner. How exactly to do that depends on the details, but often the simplest option is to use a lock. — svick, Aug 14 '14 at 09:46

TPL Dataflow: Persist previous data

1 Answers1