Turn a side-effecting function returning Option into an Iterator

Question

Assume we have a side-effecting "producer" function f: () => Option[T] which returns a Some when repeatedly called until a future point at which it will forever return None. (e.g. a wrapped Java API producing null at EOF might well have this sort of behaviour).

Is it possible to wrap this function into something like a TraversableOnce or an Iterator, with the following constraints:

Standard Scala library constructs preferred
The sequence could be arbitrarily long and holding all the values, e.g. in a Stream is not wanted
Similarly there must be no possibility of stack overflow
The visible user source code should not use a var
Thread safety is not required

There are some useful methods on the Iterator object, but nothing that exactly matches my use case. Any ideas welcome!

Régis Jean-Gilles · Accepted Answer · 2016-02-22T11:34:31.637

6

This does the trick:

def wrap[T](f: () => Option[T]): Iterator[T] = {
  Iterator.continually(f()).takeWhile(_.isDefined).flatten      
}

REPL test:

scala> :paste
// Entering paste mode (ctrl-D to finish)
var i = 0
def sideEffectingFunc(): Option[Int] = {
  i += 1
  if (i < 10) Some(i)
  else None
}
// Exiting paste mode, now interpreting.

i: Int = 0
sideEffectingFunc: ()Option[Int]

scala> val it = wrap(sideEffectingFunc)
it: Iterator[Int] = non-empty iterator

scala> it.toList
res1: List[Int] = List(1, 2, 3, 4, 5, 6, 7, 8, 9)

edited Feb 22 '16 at 11:34

answered Feb 22 '16 at 10:21

Régis Jean-Gilles

32,541
5
83
97

The final `.map(_.get)` may in recent versions of Scala be substituted by `.flatten`, which introduces an implicit TraversableOnce CBF and associated iterator. I think this still satisfies my criteria. – satyagraha Feb 22 '16 at 11:19
You're right. I missed that because the scaladoc for `Iterator` does not mention it. Seems like a scaladoc bug : `flatten` comes from the enrichement class `TraversableOnce.FlattenOps` and enrichements are supposed to be handled by scaladoc (and many are). I updated my answer. – Régis Jean-Gilles Feb 22 '16 at 11:34

score 2 · Answer 2 · answered Feb 22 '16 at 10:51

Somewhat orthogonally, this behavior can be achieved using coroutines. There is at least one library for Scala that enables coroutines, you can find it here: http://storm-enroute.com/coroutines/

Here's an example of the code you'll write to get what you want:

import org.coroutines._

def sideEffectingFunction = coroutine { () => 
  val limit = new scala.util.Random().nextInt(10)
  val seq = new scala.util.Random
  var counter = 0 // mutable state is preserved between coroutine invocations
  while (counter < limit) {
    counter += 1
    yieldval(seq.nextInt)
  }
}
defined function sideEffectingFunction

@ val cr = call(sideEffectingFunction()) 
cr: Coroutine.Frame[Int, Unit] = Coroutine.Frame<depth: 1, live: true>
@ cr.resume 
res31: Boolean = true
@ cr.value 
res32: Int = 57369026
@ cr.resume 
res33: Boolean = true
@ cr.value 
res34: Int = -1226825365
@ cr.resume 
res35: Boolean = true
@ cr.value 
res36: Int = 1304491970
@ cr.resume 
res37: Boolean = false
@ cr.value 
java.lang.RuntimeException: Coroutine has no value, because it did not yield.
  scala.sys.package$.error(package.scala:27)
  org.coroutines.Coroutine$Frame$mcI$sp.value$mcI$sp(Coroutine.scala:130)
  cmd38$.<init>(Main.scala:196)
  cmd38$.<clinit>(Main.scala:-1)

Or, alternatively:

@ val cr = call(sideEffectingFunction()) 
cr: Coroutine.Frame[Int, Unit] = Coroutine.Frame<depth: 1, live: true>
@ while(cr.resume) println(cr.value) 
-1888916682
1135466162
243385373

Or, in spirit of the previous answer:

@ val cr = call(sideEffectingFunction()) 
cr: Coroutine.Frame[Int, Unit] = Coroutine.Frame<depth: 1, live: true>
@ cr.resume 
res60: Boolean = true
@ val iter = Iterator.continually(cr.value).takeWhile(_ => cr.resume) 
iter: Iterator[Int] = non-empty iterator
@ iter.foreach(println) 
1595200585
995591197
-433181225
220387254
201795229
754208294
-363238006

The advantage of coroutines approach is that you can keep a mutable state between invocations of the underlying side-effecting functions, all nicely hidden from the outer world inside a coroutine. Coroutines can also be composed and invoke each other.

Of course, coroutines give you much more power than just getting your task to work, so this might be an overkill to add them just for this. However, it's a handy technique to be aware of.

Turn a side-effecting function returning Option into an Iterator

2 Answers2