Build an IAsyncEnumerable using TaskCompletionSource

Question

I have a method that accepts an IEnumerable and returns it transformed using the yield operator. To transform one element of the enumerable, I need to first know the value of another element. Therefore, I thought of using a TaskCompletionSource to create something like a "promise".

The problem here is that this code results in a deadlock if anything other than "a" is the value of the first TestFieldA. One solution would be to order the enumerable before passing it into the method - in which case there is no need for TaskCompletionSource altogether. I would like to know however if it can be done without this. I also know that this can be done with some LINQ queries, but this would require enumerating the input several times, which I would like to avoid.

This is is what I'm trying to achieve. (Only works if the first TestFieldA == "a")

class Test
{
    public string TestFieldA {get;set;}
    public int TestFieldB {get;set;}
}


private async IAsyncEnumerable<Test> Transform(IEnumerable<Test> inputEnumerable)
{
    var tcs = new TaskCompletionSource<int>();

    foreach(var input in inputEnumerable)
    {
        if (input.TestFieldA == "a")
        {
            tcs.SetResult(input.TestFieldB);
            yield return input;
        }
        else
        {
            input.TestFieldB -= await tcs.Task;
            yield return input;
        }
    }
}

Damien_The_Unbeliever · Answer 1 · 2020-04-30T15:20:09.640

Your current plan seems to depend on being able to travel back in time. I'd suggest just storing unsuitable items in a queue (rather than yielding them) until you find the suitable item with TestFieldA value.

At that point, you dequeue all of the queued items, use the now found value and yield each in turn. Then yield the item with the desired TestFieldA value.

How you proceed from there is slightly unclear because I don't know what you want to do if a) another a item is found and b) what to do if no a item is found.

There's no need for Task(CompletionSource), async or IAsyncEnumerable here - you cannot yield anything until you've found your a value - unless you have access to a time machine.

Bear in mind also that iterators depend on having their callers repeatedly ask for new items to make forward progress - you pause at each yield until they do. So it would be risky in the extreme to consider trying to yield items early if the yielded items have anything "Task-like" about them; The caller might decide to await on one rather than continuing the enumeration you need them to.

score 1 · Accepted Answer · answered Apr 30 '20 at 15:32

An idea could be to return an enumerable of tasks instead of an IAsyncEnumerable. Something like this:

private IEnumerable<Task<Test>> Transform(IEnumerable<Test> source)
{
    var tcs = new TaskCompletionSource<int>(
        TaskCreationOptions.RunContinuationsAsynchronously);

    foreach (var item in source)
    {
        if (item.TestFieldA == "a")
        {
            tcs.TrySetResult(item.TestFieldB);
        }
        yield return TransformItemAsync(item);
    }

    async Task<Test> TransformItemAsync(Test input)
    {
        var value = await tcs.Task.ConfigureAwait(false);
        input.TestFieldB -= value;
        return input;
    }
}

This would still create a deadlock problem if the caller attempted to await each task in sequence. To solve this problem the caller should have a way to await somehow the tasks in order of completion. There is something like that in Stephen Cleary's Nito.AsyncEx library, the extension method OrderByCompletion:

// Creates a new collection of tasks that complete in order.
public static List<Task<T>> OrderByCompletion<T>(this IEnumerable<Task<T>> @this);

You can also grab the source code from here if you want.

Thanks, that's really useful. Very close to what I'm trying to achieve! — Charalampos Detsis, Apr 30 '20 at 18:27
I am happy that I gave you ideas Charalampe! Although Damien_The_Unbeliever's [suggestion](https://stackoverflow.com/a/61525954/11178549) about buffering items until the correct one is found, is probably simpler to implement, and cheaper in terms of memory and CPU consumption. — Theodor Zoulias, Apr 30 '20 at 18:35

Build an IAsyncEnumerable using TaskCompletionSource

2 Answers2