Scala method to combine each element of an iterable with each element of another?

Question

If I have this:

val a = Array("a ","b ","c ")
val b = Array("x","y")

I would like to know if such a method exists which would let me traverse the first collection, and for each of it's elements, walk the entire second collection. For example, if we take the array a, we would have a,x,a,y,b,x,b,y,c,x,c,y. I know of zip but from what I've seen it only works on collections of the same sizes, and it associates elements from same positions.

Answer 1

I'm not sure of a "method", but this can be expressed with just a nested/compound for:

val a = Array("a ","b ","c ")
val b = Array("x","y")
for (a_ <- a; b_ <- b) yield (a_, b_)

res0: Array[(java.lang.String, java.lang.String)] = Array((a ,x), (a ,y), (b ,x), (b ,y), (c ,x), (c ,y))

Happy coding.

Answer 2

I'm using the following extensively in my code. Note that this is working for an arbitrary number of lists. It is creating an Iterator instead of a collection, so you don't have to store the potentially huge result in memory.

Any improvements are very welcome.

/**
  * An iterator, that traverses every combination of objects in a List of Lists.
  * The first Iterable will be incremented fastest. So consider the head as 
  * the "least significant" bit when counting.*/

class CombinationIterator[A](val components: List[Iterable[A]]) extends Iterator[List[A]]{
  private var state: List[BufferedIterator[A]] = components.map(_.iterator.buffered)
  private var depleted = state.exists(_.isEmpty)

  override def next(): List[A] = {
    //this function assumes, that every iterator is non-empty    
    def advance(s: List[(BufferedIterator[A],Iterable[A])]): List[(BufferedIterator[A],A)] = {
      if( s.isEmpty ){
        depleted = true
        Nil
      }
      else {
        assert(!s.head._1.isEmpty)

        //advance and return identity
        val it = s.head._1
        val next = it.next()
        if( it.hasNext){
          //we have simply incremented the head, so copy the rest
          (it,next) :: s.tail.map(t => (t._1,t._1.head))
        } else {
          //we have depleted the head iterator, reset it and increment the rest
          (s.head._2.iterator.buffered,next) :: advance(s.tail)
        }
      }
    }
    //zipping the iterables to the iterators is needed for resseting them
    val (newState, result) = advance(state.zip(components)).unzip
    
    //update state
    state = newState    
    
    result
  }

  override def hasNext = !depleted
}

So using this one, you have to write new CombinationIterator(List(a,b)) to obtain an iterator that goes through every combination.

Edit: based on user unkown's version

Note that the following version is not optimal (performance wise):

• indexed access into lists (use arrays instead)
• takeWhile evaluates after every element

.

scala> def combination(xx: List[List[_]], i: Int): List[_] = xx match {
     | case Nil => Nil
     | case x :: xs => x(i % x.length) :: combination(xs, i/x.length)
     | }
combination: (xx: List[List[_]], i: Int)List[_]

scala> def combinationIterator(ll: List[List[_]]): Iterator[List[_]] = {
     | Iterator.from(0).takeWhile(n => n < ll.map(_.length).product).map(combination(ll,_))
     | }
combinationIterator: (ll: List[List[_]])Iterator[List[_]]

scala> List(List(1,2,3),List("a","b"),List(0.1,0.2,0.3))
res0: List[List[Any]] = List(List(1, 2, 3), List(a, b), List(0.1, 0.2, 0.3))
    
scala> combinationIterator(res0)
res1: Iterator[List[_]] = non-empty iterator

scala> res1.mkString("\n")
res2: String = 
List(1, a, 0.1)
List(2, a, 0.1)
List(3, a, 0.1)
List(1, b, 0.1)
List(2, b, 0.1)
List(3, b, 0.1)
List(1, a, 0.2)
List(2, a, 0.2)
List(3, a, 0.2)
List(1, b, 0.2)
List(2, b, 0.2)
List(3, b, 0.2)
List(1, a, 0.3)
List(2, a, 0.3)
List(3, a, 0.3)
List(1, b, 0.3)
List(2, b, 0.3)
List(3, b, 0.3)

Answer 3

For a list of a unknown number of lists, of different length, and for maybe different types, you can use this:

def xproduct (xx: List [List[_]]) : List [List[_]] = 
  xx match {
    case aa :: bb :: Nil => 
      aa.map (a => bb.map (b => List (a, b))).flatten       
    case aa :: bb :: cc => 
      xproduct (bb :: cc).map (li => aa.map (a => a :: li)).flatten
    case _ => xx
}

You would call it

xproduct List (List ("a ", "b ", "c "), List ("x", "y"))

but can call it with Lists of different kind too:

scala>  xproduct (List (List ("Beatles", "Stones"), List (8, 9, 10), List ('$', '€')))  
res146: List[List[_]] = List(List(Beatles, 8, $), List(Stones, 8, $), List(Beatles, 8, €), List(Stones, 8, €), List(Beatles, 9, $), List(Stones, 9, $), List(Beatles, 9, €), List(Stones, 9, €), List(Beatles, 10, $), List(Stones, 10, $), List(Beatles, 10, €), List(Stones, 10, €))

Arrays have to be converted to Lists, and the result converted back to Arrays, if you can't use Lists.

update:

On the way towards a lazy collection, I made a functional mapping from an index (from 0 to combination-size - 1) to the result at that position, easily calculated with modulo and division, just a bit concentration is needed:

def combicount (xx: List [List[_]]): Int = (1 /: xx) (_ * _.length)

def combination (xx: List [List[_]], i: Int): List[_] = xx match {
    case Nil => Nil
    case x :: xs => x(i % x.length) :: combination (xs, i / x.length)
}

def xproduct (xx: List [List[_]]): List [List[_]] = 
  (0 until combicount (xx)).toList.map (i => combination (xx, i))

It's no problem to use a long instead, or even BigInt.

update 2, The iterator:

class Cartesian (val ll: List[List[_]]) extends Iterator [List[_]] {

  def combicount (): Int = (1 /: ll) (_ * _.length)

  val last = combicount - 1 
  var iter = 0
  
  override def hasNext (): Boolean = iter < last
  override def next (): List[_] = {
    val res = combination (ll, iter)
    iter += 1
    res
  }

  def combination (xx: List [List[_]], i: Int): List[_] = xx match {
      case Nil => Nil
      case x :: xs => x (i % x.length) :: combination (xs, i / x.length) 
  }
}

Answer 4

If you want to show off your deep knowledge of higher kinded types and category theory, you can write:

trait Applicative[App[_]] {
  def pure[A](a: A): App[A]
  def fmap[A,B](f: A => B, appA: App[A]): App[B]
  def star[A,B](appF: App[A => B], appA: App[A]): App[B]
}

object ListApplicative extends Applicative[List] {
  override def pure[A](a: A): List[A] = List(a)
  override def fmap[A,B](f: A => B, listA: List[A]): List[B] = listA.map(f)
  override def star[A,B](listF: List[A => B], listA: List[A]):List[B] = 
    for(f <- listF; a <- listA) yield f(a)
}

import ListApplicative._

def pairs[A,B](listA: List[A], listB: List[B]) = 
  star(fmap((a:A) => ((b:B) => (a,b)), listA), listB)

Other than that I would prefer pst's solution...

Answer 5

Here's one more that does the same thing as @ziggystar's last edit but doesn't use indexed access of lists.

def combinationIterator[A](xs: Iterable[Iterable[A]]): Iterator[List[A]] = {
  xs.foldRight(Iterator(List[A]())) { (heads, tails) =>
    tails.flatMap { tail =>
      heads.map(head => head :: tail)
    }
  }
}

And the sugary version:

def combinationIterator[A](xs: Iterable[Iterable[A]]): Iterator[List[A]] = {
  (xs :\ Iterator(List[A]())) { (heads, tails) =>
    for (tail <- tails; head <- heads) yield head :: tail
  }
}