Implementing a real "forall" on a list of scala futures

Question

I am using scala (2.12) futures to apply a concurrent divide and conquer approach for a complex problem. Here is some (simplified) context:

def solve(list: List[Int], constraints: Con): Future[Boolean] =
  Future.unit.flatMap{ _ =>
    //positive case
    if(list.isEmpty) Future.successful(true)

    //negative case
    else if(someTest(constraints)) Future.successful(false)

    //divide and conquer
    else {
      //split to independent problems, according to constraints
      val components: List[List[Int]] = split(list,constraints)

      //update the constraints accordingly (heavy computation here)
      val newConstr: Con = updateConstr(...)

      val futureList = components.map(c => solve(c,newConstr))
      allTrue(Future.successful(true), futureList)
    }
  }

This recursive function takes a list of integer variables and a Con object representing the problem constraints, and spawns multiple independent sub-problems during each call.

The relevant part for my question is the call to allTrue. If I was solving the problem sequentially, I would have written components.forall(c => solve(c,newConstr)). In the concurrent version however, I have something like this, which doesn't stop computation at the first false case encountered.

//async continuation passing style "forall"
def allTrue(acc: Future[Boolean], remaining: List[Future[Boolean]]): 
  Future[Boolean] = {
    remaining match {
      case Nil => acc
      case r :: tail => acc.flatMap{ b => 
        if(b) allTrue(r,tail)
        else{
          //here, it would be more efficient to stop all other Futures
          Future.successful(false)
        }
      }
    }
  }

I have read multiple blog posts and forum threads talking about how stopping scala futures is generally not a good idea, but in this case I think it would be very useful.

Any ideas on how to get the forall behaviour on a list of futures?

Why `List[Future[Boolean]]` and not `Future[List[Boolean]]`. If you want to convert you can user `Future.sequence` — Arpit Suthar, Feb 27 '18 at 12:22
I do not convert the whole list because I want to treat every element independently: the idea is that as soon as one future completes with the value `false` I want to stop every other computation and return false — H.Leger, Feb 27 '18 at 14:44
maybe you can share an atomicBoolean flag among all futures. for each future, if the flag says ok, then the future can proceed to the heavy computation part. once one future fails then the flag is set to false(by that failed future) so that other futures will skip the heavy computation part(if they haven't reached the line of code to check the flag) and return false — David, Feb 27 '18 at 15:23

score 2 · Answer 1 · edited Feb 27 '18 at 12:42

2

Simple approach without stopping futures would be Future.traverse

val all:Future[List[Boolean]] = Future.traverse(components)(c => solve(c, newConstr)
val forAll:Future[Boolean] = all.map(_.forall(identity))

For cancelable list of futures I would recommend to look at Observable pattern. In your case subscriber can unsubscribe as soon as it see False value and producer will stop calculations when no subscriber listens

edited Feb 27 '18 at 12:42

Arpit Suthar

754
1
5
19

answered Feb 27 '18 at 12:26

Nazarii Bardiuk

4,272
1
19
22

Thank you for the `traverse` tip! About the Observer pattern, I am not sure I understand.. Isn't `Future` already supposed to be a wrapper around this pattern? – H.Leger Feb 27 '18 at 15:02
Future is a single asynchronous value and Observable is a asynchronous stream of values. Have a look at reactivex introduction http://reactivex.io/intro.html – Nazarii Bardiuk Feb 28 '18 at 11:41
Ok, I undestand what you meant now. Thank you for the interesting idea, I haven't played with reactive programming yet but I will try. – H.Leger Mar 08 '18 at 09:42

Implementing a real "forall" on a list of scala futures

1 Answers1