Questions tagged [parallel-collections]
50 questions
1
vote
1 answer
Thread pool has less number of threads than what is set (in Scala)
I am using parallel collection to run some code in parallel. Here is the code
val threadIds = new ConcurrentSkipListSet[Long]()
val pool = new ForkJoinPool(250)
val forkJoinSupport = new ForkJoinTaskSupport(pool)
list.par.taskSupport =…
Raj
- 2,368
- 6
- 34
- 52
1
vote
2 answers
How to define a function which accept both Seq[T] and ParSeq[T] as parameter?
In many cases, I want to apply the same filter or map function to a Seq or ParSeq collection. However I don't want to write the code twice.
def fun(data:ParSeq[String], num_start:Int,num_end:Int) = {
data filter { x=>
val temp =…
worldterminator
- 2,968
- 6
- 33
- 52
1
vote
2 answers
Why JVM calculated PS Survivor Space size too low for parallel collector
I am using JDK1.6.0_16 JVM for the java application that is hosted on an Linux Intel procesor 80 cores machine.
while starting the Java application I have only two options configured
-Xms2048m -Xmx8000m in the JVM Options (after java command).
I…
Lokesh Garg
- 71
- 1
- 1
- 5
1
vote
3 answers
Scala, waiting to execute foreach on parallel collection
I have code like below
val g = new Graph(vertices)
//Firts part
(1 to vertices).par.foreach( i => g + new Vertex(i))
//Second part
for (i <- 1 to edges) {
val data = scala.io.StdIn.readLine()
…
matiii
- 316
- 4
- 17
1
vote
1 answer
Scala different parallel structures
I tested parallel collections on Scala vs simple collection, here is my code:
def parallelParse()
{
val adjs = wn.allSynsets(POS.ADJECTIVE).par
adjs.foreach(adj => {
parse(proc.mkDocument(adj.getGloss))
})
}
def…
Omid
- 1,959
- 25
- 42
1
vote
1 answer
Scala ParArray pattern matching
Consider
val b = ParArray("a","b","c")
However, on pattern matching b for instance as follows,
b match {
case ParArray(_,"b",_) => 2
case _ => -1
}
:11: error: object ParArray is not a case class,
nor does it…
elm
- 20,117
- 14
- 67
- 113
1
vote
1 answer
Calling map on a parallel collection via a reference to an ancestor type
I tried to make it optional to run a map operation sequentially or in parallel, for example using the following code:
val runParallel = true
val theList = List(1,2,3,4,5)
(if(runParallel) theList.par else theList) map println //Doesn't run in…
omid
- 370
- 1
- 10
1
vote
1 answer
ParMap values method alternative
ParMap (in Scala 2.9) doesn't seem to have a .values method. Why is this, and how can I work around it if I'm particularly keen on maintaining a parallel chain of processing such as the following?
myParSeq.collect{case i: Interesting =>…
Pengin
- 4,692
- 6
- 36
- 62
1
vote
1 answer
Mutating a HashMap concurrently in Scala without Actors
What I want to do is to start out using some implementation of a Map and accumulate data into it by iterating over a parallel collection. Keys can "overlap" between threads as the keys are probabilistically generated (related to random number…
adelbertc
- 7,270
- 11
- 47
- 70
0
votes
1 answer
Scala parallel collections import
ERROR: shows error on the word parallel
object parallel is not a member of package collection
import scala.collection.parallel.CollectionConverters._
:-- Already added this in build.sbt
libraryDependencies ++= {
…
0
votes
1 answer
interrupt scala parallel collection
Is there any way to interrupt a parallel collection computation in Scala?
Example:
val r = new Runnable {
override def run(): Unit = {
(1 to 3).par.foreach { _ => Thread.sleep(5000000) }
}
}
val t = new Thread(r)
t.start()
Thread.sleep(300)…
eprst
- 733
- 6
- 14
0
votes
1 answer
scala http requests using parallel collections
i'm experimenting with scala parallel collections. I'm trying to get data from a local server, that I've
set up, this is my code
val httpRequestInputs = List(inputs).par
def getResponse(data: String, url: String) = {
val request =…
chqdrian
- 335
- 4
- 17
0
votes
0 answers
Spark for non-distributed parallel computing and its performance
I am new to Spark. I am wondering how well it performs when scaled down to a single node, and how much the overhead is compared to regular non-distributed parallel approaches, so I can evaluate whether it's a good choice to write a non-distributed…
Shreck Ye
- 1,591
- 2
- 16
- 32
0
votes
1 answer
How to merge and aggregate 2 Maps in scala most efficiently?
I have the following 2 maps:
val map12:Map[(String,String),Double]=Map(("Sam","0203") -> 16216.0, ("Jam","0157") -> 50756.0, ("Pam","0129") -> 3052.0)
val map22:Map[(String,String),Double]=Map(("Jam","0157") -> 16145.0, ("Pam","0129") -> 15258.0,…
Shiv
- 3
- 2
0
votes
1 answer
Map an arithmetical operation to a Scala collection and sum the result
Here's some code I shared with my study group yesterday: https://gist.github.com/natemurthy/019e49e6f5f0d1be8719. After compiling, I run map.scala with the following heap params:
$ scala -J"-Xmx4G" map
and get the following results for 4 separate…
nmurthy
- 1,337
- 1
- 12
- 24