Scala different parallel structures

Question

I tested parallel collections on Scala vs simple collection, here is my code:

def parallelParse()
  {
    val adjs = wn.allSynsets(POS.ADJECTIVE).par
    adjs.foreach(adj => {
      parse(proc.mkDocument(adj.getGloss))
    })
  }

  def serialParse()
  {
    val adjs = wn.allSynsets(POS.ADJECTIVE)
    adjs.foreach(adj => {
      parse(proc.mkDocument(adj.getGloss))
    })
  }

The parallel collection speed up about 3 times. What other option do I have in Scala to make it even faster in parallel, I would be happy to test them and put the results here.

axel22 · Answer 1 · 2014-07-08T07:09:03.997

1

You can use futures to start asynchronous computations. You could do:

import scala.concurrent._
import scala.concurrent.duration._
import ExecutionContext.Implicits.global
val futures = wn.allSynsets(POS.ADJECTIVE).map(adj => Future {
  parse(proc.mkDocument(adj.getGloss))
})
futures.foreach(f => Await.ready(f, Duration.Inf))

Depending on the amount of work on each element in allSynsets and the number of elements in allSynsets (too many elements -> too many futures -> more overheads), you could get worse results with futures.

To ensure that you are benchmarking correctly, consider using the inline benchmarking feature of ScalaMeter 0.5:

http://scalameter.github.io/home/gettingstarted/0.5/inline/index.html

You could also use actors to achieve this, but it would require a bit more plumbing.

edited Jul 08 '14 at 07:09

answered Jul 07 '14 at 19:35

axel22

32,045
9
125
137

I used future and the benchmark, but it suddenly reports the time which is incorrect for sure, any idea? Here is the code: `val time = measure { futureParse() } println(s"Total time future: $time") def futureParse() { wn.allSynsets(POS.ADJECTIVE).foreach(adj => Future { parse(proc.mkDocument(adj.getGloss)) }) }` – Omid Jul 07 '14 at 22:25
Sorry, I must have been tired when I was writing the code example. You have to wait for the futures to complete. I've edited my answer to map each element to a `Future`, and then wait until each of the futures completes its work. – axel22 Jul 08 '14 at 07:10

Scala different parallel structures

1 Answers1