How can I await multiple futures, at once, that are nested inside a map?

Question

I have a Map<String, List<Future<Foo>>> that I want to convert to a Map<String, List<Foo>> (wait for futures to finish). Currently, this is how I perform the operation:

// source contains values with an async operation; convert to a map
final futures = source.map(
  (k, v) => MapEntry(k, v.map((e) async => await e.asyncOperation())),
);

final json = {};

// works, but must wait for each entry to process
for (final kvp in futures.entries) {
  json[kvp.key] = await Future.wait(kvp.value);
}

This will block for every entry sublist, is there a way to generate the proper Map<String, List<Foo>> while at the same time awaiting all of the inner list futures?

I can do await Future.wait(futures.values.flattened);, but how would I reassign the results back to the proper map key?

I am failing to see the issue with the solution you have proposed yourself. `await` is not a blocking operation. All of the futures you have yet to await can be completed while you `await` for one earlier in the loop to complete. All in all it should only take about as long as however long the longest future ends up taking, it is not the combined length of time of all futures to complete since they are completing concurrently. — mmcdon20, Feb 21 '22 at 19:51
Actually, looking at it again, you do not have a `Map>>` but rather a `Map>>`. This would prevent the futures from completing concurrently because the futures are being created lazily as you iterate to them in the loop. The futures later on in the loop cannot complete concurrently if they don't exist yet. If you change `MapEntry(k, v.map((e) async => await e.asyncOperation()))` to `MapEntry(k, [for (var e in v) e.asyncOperation()])`, then my above statement should hold true, since I assumed you were using a `Map>>`. — mmcdon20, Feb 21 '22 at 20:27

score 4 · Accepted Answer · answered Feb 21 '22 at 20:40

One reason to complete all the futures immediately would be that they already exist, and if you don't await all of them immediately, one of them might complete with an (unhandled!) error, which is bad.

The standard provided way to wait for more than one future at a time is Future.wait, which takes an Iterable<Future<X>> and returns a Future<List<X>>.

That will directly help you with the individual lists, but then you'll have a Future<List<Foo>> per key in the map. You'll have to convert those to a list too. So, maybe something like:

Future<Map<String, List<Foo>>> waitAll(
    Map<String, Iterable<Future<Foo>>> map) async =>
  Map.fromIterables(map.keys.toList(), await Future.wait(map.values.map(Future.wait));

jamesdlin · Answer 2 · 2022-02-21T19:59:36.847

You will want to call Future.wait<Foo> on each of the inner List<Future<Foo>>s. Each of those will return a Future<List<Foo>>; collect those Futures into their own list, and then use Future.wait again on that.

As you mentioned, the tricky part is assigning to results to the desired locations. You can use Future.then to register a completion callback for each of the Future<List<Foo>>s that assigns the resulting List<Foo> to the new Map. This is one situation where mixing Future.then with await is a little bit more straightforward than using just await. (It's still possible with await, but it's more awkward.)

For example:

import 'dart:async';

/// Randomized delays to test correctness.
final delays = <int>[for (var i = 0; i < 9; i += 1) i]..shuffle();

/// Returns `result` after a randomized delay.
Future<int> f(int result) async {
  var index = result - 1;
  assert(index >= 0);
  assert(index < delays.length);
  await Future.delayed(Duration(seconds: delays[result - 1]));
  return result;
}

var map = <String, List<Future<int>>>{
  'foo': [f(1), f(2), f(3)],
  'bar': [f(4), f(5), f(6)],
  'baz': [f(7), f(8), f(9)],
};

Future<void> main() async {
  var newMap = <String, List<int>>{};
    await Future.wait<void>([
    for (var key in map.keys)
      Future.wait<int>(map[key]!).then((list) => newMap[key] = list),
  ]);
  
  print(newMap); // Prints: {foo: [1, 2, 3], bar: [4, 5, 6], baz: [7, 8, 9]}
}

Note that the above does not guarantee that newMap preserves the order of keys in the Map. If you care about that, then an easy way to preserve the key order is to pre-add them to the new Map:

var newMap = {for (var key in map.keys) key: <int>[]};

or alternatively, remove and re-add them in the desired order afterward:

for (var key in map.keys) {
  newMap[key] = newMap.remove(key)!;
}

How can I await multiple futures, at once, that are nested inside a map?

2 Answers2