How to handle recursion with monix's observable?

Question

Using monix I'm trying to traverse a graph by building an Observable[Node] and using a breadth first algorithm. However there I have a bit of a recursion problem. Here is a snippet illustrating my problem:

package gp

import monix.eval.Task
import monix.execution.Scheduler.Implicits.global
import monix.reactive._


object HelloObservable {

  type Node = Int

  //real case fetch next node across the network so the signature 
  //has to be Node -> List[Task[Node]]
  def nexts(i : Node) : List[Task[Node]] =
    List(Task(i), Task(i+1))

  def go(i :Node) : Task[Iterator[List[Node]]] =
    Task.sequence(nexts(i).sliding(100,100).map(Task.gatherUnordered))

  def explore(r: Node): Observable[Node] = {
    val firsts = for {
      ilr <- Observable.fromTask(go(r))
      lr <- Observable.fromIterator(ilr)
      r <- Observable.fromIterable(lr)
    } yield r

    firsts ++ firsts.flatMap(explore)
  }


  def main(args : Array[String]) : Unit = {

    val obs = explore(0)

    val cancelable = obs
      .dump("O")
      .subscribe()

    scala.io.StdIn.readLine()

  }

}

The observable stop after the first iteration. Can anyone hint me why ?

score 2 · Accepted Answer · answered Dec 26 '17 at 05:43

I think the issue is not related to recursion. I think it comes from the fact that you use sliding which returns an Iterator. The major difference between Iterator and Iterable is that you can consume Iterator only once and after that all you are left with is an empty Iterator. It means when you do firsts.flatMap there is nothing left in the Observable.fromIterator(ilr) and so nothing is produced.

Fundamentally I don't think you can do a breadth-first search if you can't hold (most part of) the prefix in the memory. But since your nexts already returns List, I assume that you can afford having two copies of that list in the memory. And the second copy is a materialized result of the sliding. So your fixed code would be something like this:

object HelloObservable {

    import monix.eval.Task
    import monix.execution.Scheduler.Implicits.global
    import monix.reactive._

    type Node = Int

    //real case fetch next node across the network so the signature
    //has to be Node -> List[Task[Node]]
    def nexts(i: Node): List[Task[Node]] = List(Task(i), Task(i + 1))

    def go(i: Node): Task[List[List[Node]]] =
      Task.sequence(nexts(i).sliding(100, 100).toList.map(Task.gatherUnordered))


    def explore(r: Node): Observable[Node] = {
      val firsts = for {
        ilr <- Observable.fromTask(go(r))
        lr <- Observable.fromIterable(ilr)
        r <- Observable.fromIterable(lr)
      } yield r
      firsts ++ firsts.flatMap(explore)
    }


    def main(args: Array[String]): Unit = {

      val obs = explore(0)

      val cancelable = obs
        .dump("O")
        .subscribe()

      scala.io.StdIn.readLine()

    }
}

Thank you so much ! I feel a bit stupid XD But since my problem was not really related to the question I asked, what is the proper StackOverflow etiquette ? should I delete my question ? — lorilan, Dec 26 '17 at 16:16
@lorilan, I don't know whether such thing as an official SO etiquette exists at all :-) Probably the closest one on this topic is [this](https://stackoverflow.com/help/privileges/moderator-tools). As deleting this question will probably result in loss of reputation for me, I have a selfish motive to suggest against it. Still, I don't think that deleting this question is good idea. IMHO you should delete only irrecoverably bad questions with no good answers (i.e. pure information noise). However, if you can think of a way to improve future discoverability - you are welcome to edit your post. — SergGr, Dec 27 '17 at 01:33

How to handle recursion with monix's observable?

1 Answers1