TPL DataFlow unable to handle an exception in ActionBlock

Question

I'm trying to send a copy of one message from an ActionBlock<int> to multiple consumers which are also ActionBlock<int>. This works well, however if one of the target blocks throws an exception, it seems that this is not propagated to the source block. Here how I try to handle the exception but it never goes to the catch part:

static void Main(string[] args)
{
    var t1 = new ActionBlock<int>(async i =>
    {
        await Task.Delay(2000);
        Trace.TraceInformation($"target 1 | Thread {System.Threading.Thread.CurrentThread.ManagedThreadId} | message {i}");
    }, new ExecutionDataflowBlockOptions { BoundedCapacity = 5 });

    var t2 = new ActionBlock<int>(async i =>
    {
        await Task.Delay(1000);
        Trace.TraceInformation($"target 2 | Thread {System.Threading.Thread.CurrentThread.ManagedThreadId} | message {i}");
    }, new ExecutionDataflowBlockOptions { BoundedCapacity = 5 });

    var t3 = new ActionBlock<int>(async i =>
    {
        await Task.Delay(100);
        Trace.TraceInformation($"target 3 | Thread {System.Threading.Thread.CurrentThread.ManagedThreadId} | message {i}");
        if (i > 5)
            throw new Exception("Too big number");
    }, new ExecutionDataflowBlockOptions { BoundedCapacity = 5 });

    var targets = new [] { t1, t2, t3};

    var broadcaster = new ActionBlock<int>(
        async item =>
        {
            var processingTasks = targets.Select(async t =>
            {
                try
                {
                    await t.SendAsync(item);
                }
                catch
                {
                    Trace.TraceInformation("handled in select"); // never goes here
                }
            });

            try
            {
                await Task.WhenAll(processingTasks);
            }
            catch
            {
                Trace.TraceInformation("handled"); // never goes here
            }
        });

    for (var i = 1; i <= 10; i++)
        broadcaster.Post(i);
}

I'm not sure what I'm missing here but I would like to be able to retrive the exception and which target block has faulted.

You only `await` the `Task` from `SendAsync` which only indicates whether the item was accepted by the target. If any one of the targets throws an exception that exception will be attached to the `Completion` task of that target. In order to observe that exception you need to `await` that task, i.e. `await t3.Completion`. — JSteward, Sep 06 '17 at 22:33
An easy fix could be to replace `await t.SendAsync(item);` with `if (!await t.SendAsync(item)) await t.Completion;` That would propagate the exception out to your inner most `try/catch`. You could then throw again or add information to a new exception, e.g. which block faulted. You'd then need to handle the faulted `broadcaster` but you get the idea. — JSteward, Sep 06 '17 at 22:44
@JSteward Thanks! I've replaced with `if (!await t.SendAsync(item)) await t.Completion;` and now everything works. Post it as answer so I can accept it. — Tomasz Jaskuλa, Sep 06 '17 at 22:56

score 2 · Accepted Answer · answered Sep 07 '17 at 00:19

2

If a block enters a faulted state it will no longer accept new items and the Exception it threw will be attached to its Completion task and/or propagated with its completion if linked in a pipeline. To observe the Exception you can await the completion if the block refuses more items.

var processingTasks = targets.Select(async t =>
{
    try
    {
        if(!await t.SendAsync(item))
            await t.Completion;
    }
    catch
    {
        Trace.TraceInformation("handled in select"); // never goes here
    }
});

answered Sep 07 '17 at 00:19

JSteward

6,833
2
21
30

One question though. It seems to me that even if `t.SendAsync(item)` sends messages in order the next block after t is receving them out of order. I though DataFlow waranted the ordering of message processing? – Tomasz Jaskuλa Sep 07 '17 at 12:36
Each of your targets will receive messages in the order they are sent to your `broadcaster`. If at most one target fails the `broadcaster` will be faulted and refuse new messages. Your `await Task.WhenAll` with default block options ensures that all targets have accepted an individual message before processing the next one. Can you provide an example or use case that shows messages out of order? – JSteward Sep 07 '17 at 14:48
It was my fault. I was passing as paramater the body for `ActionBlock` as `Action` instead of `Func` – Tomasz Jaskuλa Sep 07 '17 at 15:11
Ah, glad you got it figured out. – JSteward Sep 07 '17 at 15:12
However I noticed that `if(!await t.SendAsync(item))` doesn't work anymore if I introduce another block in between. Let's say `t.SendAsync(item)` where `t` is a new block `BufferBlock` linked to the actual `ActionBlock` that throws an exception. The exception doesn't go back like before to the source so it is never catched. What is the best strategy to handle it? – Tomasz Jaskuλa Sep 07 '17 at 15:15
The root of the issue is that when a block throws an exception; that exception is attached to the blocks completion task or the completion task representing the completion of the pipeline. If `if(!await t.SendAsync(item))` where `t` represents a block that is not faulted, i.e. the new `BufferBlock`, then `SendAsync` will complete with success while the downstream `ActionBlock` is still faulted. Right now the example is not well behaved, can you post a new question with a more complete sample on [Code Review](https://codereview.stackexchange.com/) we could get things cleaned up better. – JSteward Sep 07 '17 at 15:26
Posted another question on SO: https://stackoverflow.com/questions/46104379/tpl-dataflow-and-exception-handling-in-downstream-blocks – Tomasz Jaskuλa Sep 07 '17 at 20:14

TPL DataFlow unable to handle an exception in ActionBlock

1 Answers1