Delayed Post to DataFlow

Question

When posting an item to a TPL DataFlow, is there any mechanism that can allow for a delayed post?

public partial class BasicDataFlowService
{
    private readonly ActionBlock<string> workerBlock;

    public BasicDataFlowService()
    {
        workerBlock = new ActionBlock<string>(file => DoWork(file), new ExecutionDataflowBlockOptions()
        {
            MaxDegreeOfParallelism = 32
        });
    }

    partial void DoWork(string fileName);

    private void AddToDataFlow(string file)
    {
        workerBlock.Post(file);
    }
}

Within AddToDataFlow, I would like to be able to specify a delay before the item is processed (e.g. if we've decided we want to defer the processing for 30 seconds).

I did consider using a TransFormBlock with new System.Threading.ManualResetEvent(false).WaitOne(1000);, e.g.

var requeueBlock = new TransformBlock<string, string>(file =>
{
    new System.Threading.ManualResetEvent(false).WaitOne(1000);
    return file;
});

requeueBlock.LinkTo(workerBlock);

However, this would appear to be consuming a thread needlessly that could be used by other blocks in the chain.

score 0 · Answer 1 · answered Apr 03 '17 at 21:19

First of all, you need to store the ManualResetEvent as a singleton, otherwise all the threads will got their own object to wait for, and your approach wouldn't work.

Secondly, do consider the ManualResetEventSlim version instead of heavy ManualResetEvent, if you need to do a synchronization inside one AppDomain in your pipeline.

If you want to reuse the cores of your machine without useless waiting, you should look into SpinWait lightweight structure. You may find Joseph Albahari' article useful in this case:

// singleton variable
bool _proceed;

var requeueBlock = new TransformBlock<string, string>(file =>
{
    var spinWait = new SpinWait();
    while (!_proceed)
    {
        // ensure we have the latest _proceed value
        Thread.MemoryBarrier();
        // try to spin for a while
        // after some spins, yield to another thread
        spinWait.SpinOnce();
    }
    return file;
});

SpinWait internally decides, how to yeild: with Sleep(0), Sleep(1) or Yield method calls, so it's quite effective for your case.

score 0 · Answer 2 · answered Apr 04 '17 at 15:58

To add a delay before posting a value to the workerBlock you can simply insert a delay and await it before posting the value. If your workerBlock has a bounded capacity you can await SendAsync. A few options to accomplish the goal:

private async Task AddToDataflow(string file, TimeSpan delay) {
    await Task.Delay(delay);
    await workerBlock.SendAsync(file);
}

private async Task AddToDataflow(string file) {
    var delay = TimeSpan.FromSeconds(30);
    await Task.Delay(delay);
    await workerBlock.SendAsync(file);
}

private async void AddToDataflow(string file) {
    var delay = TimeSpan.FromSeconds(30);
    await Task.Delay(delay);
    workerBlock.Post(file);
}

Delayed Post to DataFlow

2 Answers2