Using IEnumerable.Aggregate with asynchronous calls

Question

I'm trying to use the LINQ IEnumerable.Aggregate function to create a string consisting of files retrieved through async calls. Not a hundred percent sure that it's possible, and I'm also aware that there are other solutions, but I'd like to give it a try.

For now my code looks like this:

private static async Task<string> GetFiles(IEnumerable<string> filePaths)
{
    return filePaths.Aggregate(async (current, path) => current + await GetFile(path));
}

But the "async" inside the method call is error marked saying "the return of an async method must be void, Task, or Task". I get that error in general, but I'm not sure how to arrange this specific case to avoid it. Any ideas?

UPDATE: Just to clarify, the GetFile() method is indeed asynchronous and returns Task<string>:

private static async Task<string> GetFile(string filePath) { ... }

No need to get into the specific code, but for those interested it uses HttpClient.GetAsync(filePath) and the returns its response.Content.ReadAsStringAsync().Result.

`response.Content.ReadAsStringAsync().Result` is a potentional deadlock. Be careful with that. Use `await response.Content.ReadAsStringAsync();` instead. — Yuval Itzchakov, Feb 18 '15 at 14:16
Yuval: How so? Even if preceded by `var response = await client.GetAsync(filePath);` and encapsuled inside `if(response.IsSuccessStatusCode)`? — Don Simon, Feb 18 '15 at 14:34

score 11 · Accepted Answer · answered Feb 18 '15 at 14:06

11

Aggregate method won't work asynchronously. It doesn't support Task based delegates. You need to create a result sequence yourself by awaiting it in prior to call Aggregate method.

Something like this should work:

private static async Task<string> GetFiles(IEnumerable<string> filePaths)
{
    var files = filePaths
        .Select(p => GetFile(p))
        .ToArray();
    var results = await Task.WhenAll(files);

    return results
        .Aggregate((current, path) => current + path);
}

answered Feb 18 '15 at 14:06

Sriram Sakthivel

72,067
7
111
189

No need for the `ToArray`. – Yuval Itzchakov Feb 18 '15 at 14:07
@YuvalItzchakov I just add it for making the intent clear. Way of saying I intended to start the tasks right now. But you're right, It isn't needed. – Sriram Sakthivel Feb 18 '15 at 14:09
Thanks for the enlightenment and the nice solution. I edited that down to two lines, but might still go for the even simpler(?) `var result = string.Empty; filePaths.ToList().ForEach(async path => result += await GetFile(path)); return result;` – Don Simon Feb 18 '15 at 14:27
@DonSimon I don't recommend that. `List.ForEach` isn't very useful. It confuses more than it helps. Refer [this](http://blogs.msdn.com/b/ericlippert/archive/2009/05/18/foreach-vs-foreach.aspx) for more info – Sriram Sakthivel Feb 18 '15 at 14:32
@SriramSakthivel Funny, I just read that article earlier today. It pretty much says that it's no better than using a classic `foreach`, which is true, but I think it has a neater syntax in keeping it to one row. :) Which is what it comes down to here I guess, or are there any performance difference in the two solutions? – Don Simon Feb 18 '15 at 14:46
1

`List.ForEach` will be very very very little slow due to delegate invocations. But that's negligible. – Sriram Sakthivel Feb 18 '15 at 14:47

Yuval Itzchakov · Answer 2 · 2015-02-18T14:18:52.227

9

As @Sriram said, LINQ and async-await don't work that well together because there's no built-in support for async Task delegates.

What you can do is create an async overload of aggregate yourself:

public static class AsynchronousEnumerable
{
    public static async Task<TSource> AggregateAsync<TSource>
                                      (this IEnumerable<TSource> source,
                                       Func<TSource, TSource, Task<TSource>> func)
    {
       using (IEnumerator<TSource> e = source.GetEnumerator())
       {
            if (!e.MoveNext())
            {
                throw new InvalidOperationException("Sequence contains no elements");
            }

            TSource result = e.Current;
            while (e.MoveNext()) result = await func(result, e.Current);
            return result;
        }
    }
}

And now you can do the following:

private static Task<string> GetFiles(IEnumerable<string> filePaths)
{
    return filePaths.AggregateAsync(async (current, path) => current + 
                                                             await GetFile(path));
}

edited Feb 18 '15 at 14:18

answered Feb 18 '15 at 14:13

Yuval Itzchakov

146,575
32
257
321

2

Good attempt. There is a bug in your code. You can't access `e.Current` blindly without calling `e.MoveNext()` and checking whether it returns true or not. I hope you'll fix it. +1 in advance. – Sriram Sakthivel Feb 18 '15 at 14:17
@SriramSakthivel There's the `while (e.MoveNext())`. I've added brackets around it to make it clear. – Yuval Itzchakov Feb 18 '15 at 14:18
1

This a bit overkill for my specific case, but wow, that's very neat! +1 – Don Simon Feb 18 '15 at 14:19
Nope, I'm talking about the above line to that while. `TSource result = e.Current;` – Sriram Sakthivel Feb 18 '15 at 14:19
@SriramSakthivel Right, fixed that. Thanks. – Yuval Itzchakov Feb 18 '15 at 14:23

score 3 · Answer 3 · edited Sep 24 '15 at 07:39

If you want to use async inside Aggregate, you must realize that anything asynchronous always returns a Task. Taking this into account, it becomes obvious that the result of your Aggregate call should therefore also be a Task.

For example, calculating the sum of a collection of numbers that are returned asynchronously:

private static async Task<int> GetSumAsync(IEnumerable<Task<int>> numbers) {
  return await numbers
    .Aggregate(Task.FromResult(0), async (sumSoFar, nextNumber) => (await sumSoFar) + (await nextNumber));
}

I am a little confused as to what you're exactly hoping to do with your GetFiles method. You do realize that Aggregate reduces a collection to just one thing, right? (The 'sum' function is a good example)

Using IEnumerable.Aggregate with asynchronous calls

3 Answers3