How to await multiple IAsyncEnumerable

Question

We have code like this:

var intList = new List<int>{1,2,3};
var asyncEnumerables = intList.Select(Foo);

private async IAsyncEnumerable<int> Foo(int a)
{
  while (true)
  {
    await Task.Delay(5000);
    yield return a;
  } 
}

I need to start await foreach for every asyncEnumerable's entry. Every loop iteration should wait each other, and when every iteration is done i need to collect every iteration's data and process that by another method.

Can i somehow achieve that by TPL? Otherwise, couldn't you give me some ideas?

What do you *actually* want to do? You could use the operators in [System.Linq.Async](https://www.nuget.org/packages/System.Linq.Async) like Concat or Zip to combine multiple async streams into one, and use eg Aggregate or Sum to process the data — Panagiotis Kanavos, Dec 27 '19 at 15:40
You could adapt the code for [Zip](https://github.com/dotnet/reactive/blob/master/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/Zip.cs) for example to handle more than two async streams, or use Zip multiple times — Panagiotis Kanavos, Dec 27 '19 at 15:43
So the expected output is an `IAsyncEnumerable`, that yields the array [1, 2, 3] every 15,000 msec? — Theodor Zoulias, Dec 27 '19 at 20:05
And why do you want to use foreach? Just because it is available, doesn't mean it will suit all scenarios. IMO it would be much easier to use simple `GetAsyncEnumerator` and work with enumerator in this case. — Krzysztof, Dec 27 '19 at 22:26

Vladislav Shirshakov · Answer 1 · 2019-12-27T22:08:06.333

6

What works for me is the Zip function in this repo (81 line)

I'm using it like this

var intList = new List<int> { 1, 2, 3 };
var asyncEnumerables = intList.Select(RunAsyncIterations);
var enumerableToIterate = async_enumerable_dotnet.AsyncEnumerable.Zip(s => s, asyncEnumerables.ToArray());

await foreach (int[] enumerablesConcatenation in enumerableToIterate)
{
    Console.WriteLine(enumerablesConcatenation.Sum()); //Sum returns 6
    await Task.Delay(2000);
}

static async IAsyncEnumerable<int> RunAsyncIterations(int i)
{
    while (true)
        yield return i;
}

edited Dec 27 '19 at 22:08

answered Dec 27 '19 at 21:58

Vladislav Shirshakov

192
3
17

I reviewed that implementation, and I observed [a couple of issues](https://github.com/akarnokd/async-enumerable-dotnet/issues/50). – Theodor Zoulias Jun 26 '23 at 09:22

Theodor Zoulias · Answer 2 · 2023-06-26T13:18:12.293

Here is a generic method Zip that you could use, implemented as an iterator. The cancellationToken is decorated with the EnumeratorCancellation attribute, so that the resulting IAsyncEnumerable is WithCancellation friendly.

using System.Runtime.CompilerServices;

public static async IAsyncEnumerable<TSource[]> Zip<TSource>(
    IEnumerable<IAsyncEnumerable<TSource>> sources,
    [EnumeratorCancellation]CancellationToken cancellationToken = default)
{
    var enumerators = sources
        .Select(x => x.GetAsyncEnumerator(cancellationToken))
        .ToArray();
    try
    {
        while (true)
        {
            var array = new TSource[enumerators.Length];
            for (int i = 0; i < enumerators.Length; i++)
            {
                if (!await enumerators[i].MoveNextAsync().ConfigureAwait(false))
                    yield break;
                array[i] = enumerators[i].Current;
            }
            yield return array;
        }
    }
    finally
    {
        foreach (var enumerator in enumerators)
        {
            await enumerator.DisposeAsync().ConfigureAwait(false);
        }
    }
}

Usage example:

await foreach (int[] result in Zip(asyncEnumerables))
{
    Console.WriteLine($"Result: {String.Join(", ", result)}");
}

This implementation is not concurrent. The MoveNextAsync operations are launched subsequently, the one after the completion of the other. Making a concurrent implementation (ZipConcurrent) is possible, and shouldn't be particularly difficult, but it should be done with caution.

This implementation is also not 100% robust. It assumes that no GetAsyncEnumerator call can fail, and no DisposeAsync call can fail synchronously or asynchronously, otherwise some enumerators might be left undisposed. These are reasonable assumptions, but a perfectly robust implementation shouldn't rely on any of those.

How to await multiple IAsyncEnumerable

2 Answers2