0

I've tried to model the situation I have in a real code.

The use case is simple: There is:

 Future {blocking { 
   // in there it calls nested futures    
 }}

More detailed (global Execution context is used as parent one, but there is was attemp to use separete one for children):

object ExContexts {
  val main = scala.concurrent.ExecutionContext.Implicits.global
  val ecChildren = scala.concurrent.ExecutionContext.fromExecutor(Executors.newFixedThreadPool(1))
}

object NestedFutures2 extends App {

  import scala.concurrent.ExecutionContext.Implicits.global

  val cores = Runtime.getRuntime.availableProcessors

  val works = parentWork(ExContexts.main) // main EC

  val result1: Seq[Seq[Future[String]]] = Await.result(Future.sequence(works), Duration.Inf)
  println("parents are done their work")
  val result2: Seq[String] = Await.result(Future.sequence(result1.flatten), Duration.Inf)

  // ---

  def parentWork(ex:ExecutionContext): Seq[Future[Seq[Future[String]]]] = {

    val works: Seq[Future[Seq[Future[String]]]] =

      (1 to cores * 2) map { x =>

        Future {
          blocking { // will create new thread/place if needed

            val parentName = Thread.currentThread.getName

            println("parent: " + parentName + " started an action")

            val playFutureOutcomes: Seq[Future[String]] = (1 to 10) map {stuffId =>
              childPlay(parentName = parentName)(ExContexts.ecChildren)
            }

            Thread.sleep(1000)

            println(s"[${timeStamp()}] parent: " + parentName + " has finished the action")

            playFutureOutcomes
          }
        }
      }
    works
  }

  def childPlay(parentName:String)(ex:ExecutionContext):Future[String] = {
    Future {
      Thread.sleep(2000) // two seconds play session
      val threadName = Thread.currentThread.getName
      // log
      println("child: " + threadName + " of " + parentName + " parent")
      Thread.currentThread.getName
    }
  }

  def timeStamp(pattern:String = "ss:mm : hh"): String = {
    val fmt = DateTimeFormat.forPattern(pattern)
    fmt.print(DateTime.now)
  }


}

From the output (5 of 8 parrent futures are done) I see that children make use of parents threads...:

[01:31 : 05] parent: ForkJoinPool-1-worker-5 has finished the action
[01:31 : 05] parent: ForkJoinPool-1-worker-11 has finished the action
[01:31 : 05] parent: ForkJoinPool-1-worker-1 has finished the action
[01:31 : 05] parent: ForkJoinPool-1-worker-3 has finished the action
[01:31 : 05] parent: ForkJoinPool-1-worker-7 has finished the action
child: ForkJoinPool-1-worker-13 of ForkJoinPool-1-worker-1 parent
child: ForkJoinPool-1-worker-5 of ForkJoinPool-1-worker-5 parent
child: ForkJoinPool-1-worker-1 of ForkJoinPool-1-worker-1 parent
child: ForkJoinPool-1-worker-11 of ForkJoinPool-1-worker-11 parent
child: ForkJoinPool-1-worker-3 of ForkJoinPool-1-worker-3 parent
child: ForkJoinPool-1-worker-7 of ForkJoinPool-1-worker-7 parent
child: ForkJoinPool-1-worker-13 of ForkJoinPool-1-worker-11 parent
...

I expect parents one to done their work in ~1 second as it was if we don't have childrent. Also I would expect to see two connecton pools instead of one.

Q: what would be the suggestion to achive it?

UPDATE: I guess it is all about Join-Pool based upon Implicit.glocal is built. All nested Furue will be joined to one pool?

// What (I think) I see is that if we have 'global' execution context as parent/main,
// then whatever we pass as execution context to the child Future still will use global context
// and still will be using ONE connection pool.
// And then therefore wll have performance problem if children do not use 'blocking' -
// parents will be stuck in cores-long connection pool

// So: Children will 'inherit' execution context, but not 'blocking'
// and as children connected/joined to the execution context the parents will not be able to do
// anything about it but wait for the free space in the pool (when child is done)
// new additional threads will not be creating for the next parent because (i guess)
// Global execution context will not see the reason of/for it.

// Like a snapshot of the moment of time:
// Connection Pool: [ch1, ch2, ch3, p1]

// It things:Ok there is one parent (p1) in the pool if one more parent
// future will come I create for it new thread it the pool.
// But because we have more joined children than parents in the list
// the moment when Global ex context can help parent will never come.

Should then I not to use that global context if I want parents to be independant on childrent and vice versa? (funny how many people in Internet are playing with global only.)

Say I know ma app has 20 000 clients and each of them makes 1-100 sec long operations (op1->op1.op1) all the time. Then I just could dedicate one executon context with 20 000 connecton pool (max). And as many execution context as needed for all sort of long-running operations (parent->child where child is not reportng to the parent). Say: printingExCon, logExCon, readStufExCon, writeStuffExCon

But if I know that one oparation should include anoter one and they maybe joined then (only) I may use that global Ex context?

Otherwise if I would use global ex context as may man context - all and everyone would join to it and I would end up having performance problem that I explained at te beginning of this Sunday post..

UPDATE: I wasted good chunk of time trying to understand why fixed version (with 'ex' passed to childPlay's Future{}(ex) was producing an output that ignore sleeps and behaves weired on many other cases. The problem was fixed when instead of running/buildng in IntellyJIDEA 14 (by CTRL-SHIFT+F10) i build it wit sbt run..

ses
  • 13,174
  • 31
  • 123
  • 226
  • there is an issue in your childPlay function - you should create future with proper context `Future { ... }(ex)`. That will print you what you expect `child: pool-2-thread-1 of ForkJoinPool-1-worker-7 parent` – Vladimir Petrosyan Aug 25 '15 at 09:31
  • yep. or I should have added "implicit" to "ex"in child function as a parameter. – ses Aug 25 '15 at 11:19

0 Answers0