Lazy eager map evaluation

Question

There are basically two options to evaluate a map in Scala.

• Lazy evaluation computers the function that is passed as a parameter when the next value is needed. IF the function takes one hour to execute then it's one hour to wait when the value is needed. (e.g. Stream and Iterator)
• Eager evaluation computes the function when the map is defined. It produces a new list (Vector or whatever) and stores the results, making the program to be busy in that time.
• With Future we can obtain the list (Seq or whatever) in a separate thread, this means that our thread doesn't block, but the results have to be stored.

So I did something different, please check it here.

This was a while ago so I don't remember whether I tested it. The point is to have a map that applies concurrently (non-blocking) and kind of eagerly to a set of elements, filling a buffer (the size of the number of cores in the computer, and not more). This means that:

1. The invocation of the map doesn't block the current thread.
2. Obtaining an element doesn't block the current thread (in case there was time to calculate it before and store the result in the buffer).
3. Infinite lists can be handled because we only prefetch a few results (about 8, depending on the number of cores).

So this all sounds very good and you may be wondering what's the problem. The problem is that this solution is not particularly elegant IMHO. Assuming the code I shared works in Java and or Scala, to iterate over the elements in the iterable produced by the map I would only need to write:

new CFMap(whateverFunction).apply(whateverIterable)

However what I would like to write is something like:

whateverIterable.bmap(whateverFunction)

As it is usual in Scala (the 'b' is for buffered), or perhaps something like:

whateverThing.toBuffered.map(whateverFunction)

Either of them works for me. So the question is, how can I do this in an idiomatic way in Scala? Some options:

• Monads: create a new collection "Buffered" so that I can use the toBuffered method (that should be added to the previous ones as an implicit) and implement map and everything else for this Buffered thing (sounds like quite some work).
• Implicits: create an implicit method that transforms the usual collections or the superclass of them (I'm not sure about which one should it be, Iterable maybe?) to something else to which I can apply the .bmap method and obtain something from it, probably an iterable.
• Other: there are probably many options I have not considered so far. It's possible that some library does already implement this (I'd be actually surprised of the opposite, I can't believe nobody thought of this before). Using something that has already been done is usually a good idea.

Please let me know if something is unclear.

Answer 1

What you're looking for is the "pimp-my-library" pattern. Check it out:

object CFMapExtensions {
  import sanity.commons.functional.CFMap
  import scala.collection.JavaConversions._

  implicit class IterableExtensions[I](i: Iterable[I]) {
    def bmap[O](f: Function1[I, O]): Iterable[O] = new CFMap(f).apply(asJavaIterable(i))
  }

  implicit class JavaIterableExtensions[I](i: java.lang.Iterable[I]) {
    def bmap[O](f: Function1[I, O]): Iterable[O] = new CFMap(f).apply(i)
  }

  // Add an implicit conversion to a java function.
  import java.util.function.{Function => JFunction}
  implicit def toJFunction[I, O](f: Function1[I, O]): JFunction[I, O] = {
    new JFunction[I, O]() {
      def apply(t: I): O = f(t)
    }
  }
}

object Test extends App {
  import CFMapExtensions._
  List(1,2,3,4).bmap(_ + 5).foreach(println)
}