Splitting an array at random in Scala

Question

I'm building a decision tree system in Scala, but some of the entries in my data have identical attributes. I've gotten around this by implementing a "random" node type, allowing the query to randomly select which branch to traverse, but I'm getting a "MatchError" when trying to split the remaining examples at random. My current code:

def splitRandom(examples: Array[String]): Array[String]={
        examples.collect {case x if(r.nextInt(100) < 50) => x}
}

"examples" is an array of strings, with each string being a line containing a single data entry with all of its attributes.

Rex Kerr · Accepted Answer · 2012-08-23T19:32:50.473

7

collect isn't a good choice for random behavior because the same condition can be evaluated twice (first on an isDefinedAt, and then a second time to compute the value); if it says true the first time and false the second--on the same input--match will be upset. Use filter instead:

examples.filter(_ => r.nextInt(100) < 50)

edited Aug 23 '12 at 19:32

answered Aug 23 '12 at 14:50

Rex Kerr

166,841
26
322
407

Had to remove the `if` but other than that very helpful. Thanks! – Tin Wizard Aug 23 '12 at 15:09
@Amadeus9 - Oops. Fixed now. I assume you might pick values other than 50, which is why you're using `nextInt`? – Rex Kerr Aug 23 '12 at 19:33
Yeah. For now I'm having it pick without bias, but I may end up changing that. – Tin Wizard Aug 26 '12 at 20:19

score 0 · Answer 2 · answered Nov 16 '15 at 09:48

there is a solution fits your issue:

import util.Random
val shuffled = Random.shuffle(your_array)
val (first, second) = shuffled.splitAt(your_position)

I found this trick when I wanted a rdd.randomSplit's counterpart for Scala List or Array

You can do some type transformation if needed

Splitting an array at random in Scala

2 Answers2