Scala for comprehension with future and options

Question

object Main extends App {
  val p1 = Promise[Option[String]]()
  val p2 = Promise[Option[String]]()
  val f1 = p1.future
  val f2 = p2.future

  val res = (for{
    file1Opt <- f1
    file2Opt <- f2
    file1 <- file1Opt
    file2 <- file2Opt
  } yield {
    combineFiles(file1, file2)
  }).fallbackTo(Future.successful("Files not found"))

  Thread.sleep(2000)
  println("XXXXXXXXXXXXXXXXXXX")

  p1.success(Some("file one"))
  p2.success(Some("file two"))

  val finalData = res.map(s =>
    s + " " + "add more data to the file"
  ) 

  finalData.map(println(_))

  def combineFiles(f1: String, f2: String): String = {
    f1 + " " + f2
  }
}

I have two functions that return Future[Option[String]] and I need to combine the two strings into one string.

I want the output to be either combination of two strings and footer: "file one file two add more data to the file" or default when one or both of the Futures return None: "Files not found add more data to file".

How can this be achieved?

Compiler error:

Error:(16, 11) type mismatch;
found   : Option[String]
required: scala.concurrent.Future[?]
file1 <- file1Opt
      ^ 

Answer 1

Well, without doing anything fancy like monad transformers or stuff, one can just simply nest for comprehensions. It will be more wordy, but no extra dependencies.

val res = (for{ 
  file1Opt <- f1
  file2Opt <- f2
} yield for {
  file1 <- file1Opt
  file2 <- file2Opt
} yield combineFiles(file1, file2))
.fallbackTo(Future.successful(Some("Files not found")))
//or, alternatively, .fallbackTo(Future.successful(None))

Ultimately, the problem here is that you try to combine Future and Option in a single for comprehension. That just doesn't work for the reasons other respondents mentioned. Nesting, however, works just fine.

The downside of the nesting is that you end up with very complex data structures, which might not be easy to use elsewhere in your program. You should think about how you would flatten them, i.e. going from Future[Option[String]] to just Future[String]. Is your particular case you can do something like this: res.map(_.getOrElse("")).

Okay, may 2 levels of nesting is fine, but you nest more than that, consider flattening that hierarchy before letting your colleagues deal with it. :)

Answer 2

Like alf mentioned in his answer, you can use monad tranformers for this, in this case OptionT.

An example using cats :

import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
import cats.data.OptionT
import cats.implicits._

val file1: Future[Option[String]] = Future.successful(Some("file1"))
val file2: Future[Option[String]] = Future.successful(Some("file2"))

val combinedOT: OptionT[Future, String] =
  for {
    f1 <- OptionT(file1)
    f2 <- OptionT(file2)
  } yield s"$f1 $f2"

val combinedFO: Future[Option[String]] = combinedOT.value
val combinedF: Future[String] = combinedOT.getOrElse("Files not found")

Note that if you use cats, you can replace the for comprehension in combinedOT2 by using a cartesian builder (the |@|), because file2 doesn't depend on file1 :

val combinedOT2: Future[Option[String]] = 
  (OptionT(file1) |@| OptionT(file2)).map(_ + " " + _).value

You can still use fallbackTo if the "combined" Future fails, eventhough it is probably better to use recover or recoverWith to actually check which Throwables you want to recover from.

Answer 3

I think exactly this problem is covered in this 47deg blog post, as well as in this one: monads do not compose, so you need a transformer from one monad to another, as there's no flatMap operation that would (flat) map a Future into Option.