7

I would like to know what is the recommended way to execute multiple async methods in parallel?

in System.Threading.Tasks.Dataflow we can specify the max degree of parallelism but unbounded is probably the default for Task.WhenAll too ?

this :

var tasks = new List<Task>();
foreach(var item in items)
{
    tasks.Add(myAsyncMethod(item));
}
await Task.WhenAll(tasks.ToArray());

or that :

var action = new ActionBlock<string>(myAsyncMethod, new ExecutionDataflowBlockOptions
        {
            MaxDegreeOfParallelism = DataflowBlockOptions.Unbounded,
            BoundedCapacity = DataflowBlockOptions.Unbounded,
            MaxMessagesPerTask = DataflowBlockOptions.Unbounded
        });
foreach (var item in items) { }
{
     action.Post(item);
}
action.Complete();

await action.Completion;
fred_
  • 1,486
  • 1
  • 19
  • 31
  • `Parallel.ForEach()`? – Dmitry Bychenko May 16 '16 at 09:28
  • 2
    Parallel and async doesn't mix so well, and the methods are async (I cannot change them) – fred_ May 16 '16 at 09:34
  • 1
    The `Task.WhenAll` code you've shown has nothing to do with executing tasks in parallel - any decisions about the use of e.g. thread pool threads, parallelism, etc, is happening within `myAsyncMethod` which is (presumably) returning hot `Task`s. – Damien_The_Unbeliever May 16 '16 at 09:49
  • Just a note. `BoundedCapacity = DataflowBlockOptions.Unbounded` is the default. By including it explicitly in the options you open the possibility that you, or someone else, in the future decides to configure it to a better value. And then your program will start behaving oddly, and you'll spent a day and a half trying to figure out why. – Theodor Zoulias Oct 30 '19 at 12:01
  • 1
    The `Post` method drops items to the floor when the receiving block refuses to accept them because its buffer has become full. To avoid this unpleasant situation, it is better to be proactive and use `await action.SendAsync(item)`, or `action.SendAsync(item).Wait()` if you are not inside an asynchronous method. The overhead of [`SendAsync`](https://learn.microsoft.com/en-us/dotnet/api/system.threading.tasks.dataflow.dataflowblock.sendasync) is negligible. – Theodor Zoulias Oct 30 '19 at 12:01

2 Answers2

6

I would like to know what is the recommended way to execute multiple async methods in parallel?

Side note: actually not parallel, but concurrent.

in System.Threading.Tasks.Dataflow we can specify the max degree of parallelism but unbounded is probably the default for Task.WhenAll too ?

As someone commented, Task.WhenAll only joins existing tasks; by the time your code gets to Task.WhenAll, all the concurrency decsions have already been made.

You can throttle plain asynchronous code by using something like SemaphoreSlim.

The decision of whether to use asynchronous concurrency directly or TPL Dataflow is dependent on the surrounding code. If this concurrent operation is just called once asynchronously, then asynchronous concurrency is the best bet; but if this concurrent operation is part of a "pipeline" for your data, then TPL Dataflow may be a better fit.

Stephen Cleary
  • 437,863
  • 77
  • 675
  • 810
2

Both methods are acceptable and the choice should be governed by your requirements as you can see Dataflow gives you a lot of configurability that you would otherwise have to implement manually when using Tasks directly.

Note that in both situations the Task Pool will be responsible for enqueuing and running the tasks so the behaviour should remain the same.

Dataflow is good at chaining together groups of composable asynchronous operations whereas using tasks gives you finer grained control.

Slugart
  • 4,535
  • 24
  • 32