Awaiting all Scala Futures to complete

Question

I have a for comprehension like this using an image library that supports concurrent operations https://github.com/sksamuel/scrimage :

for (
      file <- myDirectory.listFiles;
      image <- AsyncImage(file);
      scaled <- image.scale(0.5)
      // possibly more ops here? 
    )
    {
      scaled.writer(Format.PNG).write(new File(dirOutput + file.getName))
    }

here is the definition for write():

def write(out: OutputStream)(implicit executionContext: ExecutionContext): Future[Unit] = Future {
  writer.write(out)
}

What I want to do is wait until all my images in my directory finish resizing before my program shutdowns. From what I gleaned from other posts on SO is to basically stuff all these Futures into a list of Futures and and use Await for that to finish... Can anyone help me out here?

My initial thought was to use a countdown latch for the total number of images and use that to block the main thread, but I don't think thats inline with the Scala Futures way of thinking — bonez, Jun 05 '15 at 21:19
Are you looking for `Future.sequence` ? [Future.sequence](http://www.scala-lang.org/api/current/index.html#scala.concurrent.Future$@sequence[A,M[X]%3C:TraversableOnce[X]]%28in:M[scala.concurrent.Future[A]]%29%28implicitcbf:scala.collection.generic.CanBuildFrom[M[scala.concurrent.Future[A]],A,M[A]],implicitexecutor:scala.concurrent.ExecutionContext%29:scala.concurrent.Future[M[A]]) — jilen, Jun 05 '15 at 21:20
@jilen possibly .. I tired playing with that, but couldn't get it working, maybe you could show me how to use it here? — bonez, Jun 05 '15 at 21:21
`val futs = for {xxxx} yield {scaled.writer()....} ; Future.sequence(futs)`, Something like this will work. You seems not `yield` the future here — jilen, Jun 05 '15 at 21:23
I tried that and got [error] found : scala.concurrent.Future[scala.concurrent.Future[Unit]] [error] required: scala.collection.GenTraversableOnce[?] [error] image <- AsyncImage(file); I have no idea ?? — bonez, Jun 05 '15 at 21:28

Arne Claassen · Accepted Answer · 2015-06-05T23:51:58.963

The problem you are encountering is that you are mixing two different monadic comprehensions, i.e. listFiles returns a List[File] while image.scale returns a Future[Image]. While both can be used in a for-comprehension, they cannot be mixed. Also, image <- AsyncImage(file) should be an assignment, since it just returns an instance:

// start the scaling operations (this is a List comprehension)
val scaleOps: List[Future[Unit]] = for {
  file <- myDirectory.listFiles
  image = AsyncImage(file)

  // Now transform the image into a Future[Unit] with a nested Future comprehension
  ops = for {
    scaled <- image.scale(0.5)
    written <- scaled.writer(Format.PNG).write(new File(dirOutput + file.getName)) 
  } yield { written }
} yield { ops }

// collect and await the results
Await.result(Future.sequence(scaleOps), 10 minutes)

In the outer comprehension <- always produces an item from a List and yield returns the final list.

In the inner comprehension <- always produces the value of the Future, allowing you to use it as input for another call producing a Future, hence producing a Future[Unit] out of the image.

Awaiting all Scala Futures to complete

1 Answers1