Is there a safe way in Scala to transpose a List of unequal-length Lists?

Question

Given the following List:

val l = List(List(1, 2, 3), List(4, 5), List(6, 7, 8))

If I try to transpose it, Scala will throw the following error:

scala> List.transpose(l)
java.util.NoSuchElementException: head of empty list
    at scala.Nil$.head(List.scala:1365)
    at scala.Nil$.head(List.scala:1362)
    at scala.List$$anonfun$transpose$1.apply(List.scala:417)
    at scala.List$$anonfun$transpose$1.apply(List.scala:417)
    at scala.List.map(List.scala:812)
    at scala.List$.transpose(List.scala:417)
    at .<init>(<console>:6)
    at .<clinit>(<console>)
    at RequestResult...

This is because List.transpose assumes equal-length lists and so uses the head method:

def transpose[A](xss: List[List[A]]): List[List[A]] = {
  val buf = new ListBuffer[List[A]]
  var yss = xss
  while (!yss.head.isEmpty) {
    buf += (yss map (_.head))
    yss = (yss map (_.tail))
  }
  buf.toList
}

I would like to get the following:

List(List(1, 4, 6), List(2, 5, 7), List(3, 8))

Is writing my own version of transpose the best way to do this? This is what I came up with:

def myTranspose[A](xss: List[List[A]]): List[List[A]] = {
  val buf = new ListBuffer[List[A]]
  var yss = xss
  while (!yss.head.isEmpty) {
    buf += (yss filter (!_.isEmpty) map (_.head))
    yss = (yss filter (!_.isEmpty) map (_.tail))
  }
  buf.toList
}

Update: I was interested in comparing the speed of the different solutions offered here, so I put together the following little benchmark:

import scala.testing.Benchmark
import scala.collection.mutable.ListBuffer

trait Transpose extends Benchmark {
  def transpose[Int](xss: List[List[Int]]): List[List[Int]] = Nil
  val list: List[List[Int]] = List(List(1,2,3), Nil, List(4,5,99,100), List(6,7,8))
  def run = {
    val l = transpose(list)
    println(l)
    l
  }
}

object PRTranspose extends Transpose {
  override def transpose[Int](xss: List[List[Int]]): List[List[Int]] = {
    val buf = new ListBuffer[List[Int]]
    var yss = xss
    while (!yss.head.isEmpty) {
      buf += (yss filter (!_.isEmpty) map (_.head))
      yss = (yss filter (!_.isEmpty) map (_.tail))
    }
    buf.toList
  }
}

object ACTranspose extends Transpose {
  override def transpose[Int](xss: List[List[Int]]): List[List[Int]] = {
    val b = new ListBuffer[List[Int]]
    var y = xss filter (!_.isEmpty)
    while (!y.isEmpty) {
      b += y map (_.head)
      y = y map (_.tail) filter (!_.isEmpty)
    }
    b.toList
  }
}

object ETranspose extends Transpose {
  override def transpose[Int](xss: List[List[Int]]): List[List[Int]] = xss.filter(!_.isEmpty) match {    
    case Nil => Nil
    case ys: List[List[Int]] => ys.map{ _.head }::transpose(ys.map{ _.tail })
  }
}

My commands were:

scala PFTranspose 5 out.log
scala ACTranspose 5 out.log
scala ETranspose 5 out.log

My results were:

PRTranspose$            10              0               1               1               0
ACTranspose$            9               2               0               0               0
ETranspose$             9               3               2               3               1

Do you intend to handle the case where the first list (List(1,2,3)) of the input is not the max size of all the lists. E.g. how do you handle input of List(List(1,2,3), List(4,5,99,100), List(6,7,8)) ? — Mitch Blevins, Nov 05 '09 at 23:28
But, it does have a bug if the first list isn't at least as great as any other. — Daniel C. Sobral, Nov 06 '09 at 01:02
Good question. In my specific case, the order of the contents in the subsequent sublists doesn't matter, so sorting the input list's lists by length works: myTranspose(l.sort((a, b) => a.length > b.length)) — pr1001, Nov 06 '09 at 01:16
Counting on an undocumented idiosyncrasy of the implementation like "it does what I want if the lists are sorted longest to shortest" is not something I recommend. And, that approach is now broken. — psp, Feb 20 '11 at 08:01

score 10 · Accepted Answer · answered Nov 06 '09 at 01:23

How about this:

    scala> def transpose[A](xs: List[List[A]]): List[List[A]] = xs.filter(_.nonEmpty) match {    
         |     case Nil    =>  Nil
         |     case ys: List[List[A]] => ys.map{ _.head }::transpose(ys.map{ _.tail })
         | }
    warning: there were unchecked warnings; re-run with -unchecked for details
    transpose: [A](xs: List[List[A]])List[List[A]]

    scala> val ls = List(List(1, 2, 3), List(4, 5), List(6, 7, 8))
    ls: List[List[Int]] = List(List(1, 2, 3), List(4, 5), List(6, 7, 8))

    scala> transpose(ls)
    res0: List[List[Int]] = List(List(1, 4, 6), List(2, 5, 7), List(3, 8))

    scala> val xs = List(List(1,2,3), List(4,5,99,100), List(6,7,8))
xs: List[List[Int]] = List(List(1, 2, 3), List(4, 5, 99, 100), List(6, 7, 8))

scala> transpose(xs)
res1: List[List[Int]] = List(List(1, 4, 6), List(2, 5, 7), List(3, 99, 8), List(100))

rats. i was looking for that but got muddled up and ran out of time. i prefer it to mine. — andrew cooke, Nov 06 '09 at 09:43
How can this be generalized to taking groups of a fixed size (rather than of size 1 like here). So if the group size is 2, the result for the first example would be `List(List(1,2,4,5,6,7), List(3,8))` and the second example would be `List(List(1,2,4,5,6,7), List(3,99,100,8))` — ved, Feb 28 '20 at 19:27

score 4 · Answer 2 · edited May 23 '17 at 12:33

I suspect the reason transpose is not defined on a "non-rectangular" list of lists is because mathematically the transpose operation is well-defined only on "rectangular structures". A desirable property of a transpose operation is that transpose( transpose(x) ) == x. This is not the case in your generalization of the transpose operation on non-rectangular list of lists.

Also, take a look at my post on Transposing arbitrary collection-of-collections in Scala and think about doing it for non-rectangular collections-of-collections. You will end up with mathematically inconsistent definitions, leave alone implementations.

I do agree that idiosyncratic "transpose" operations are often useful, but I also think that they should not be made available in standard libraries because of potential confusion regarding their precise definitions.

andrew cooke · Answer 3 · 2012-12-07T16:59:46.743

3

I don't know of (and can't imagine - isn't this is a bit odd?! [see discussion in comments]) a library function, but I can polish the code a little:

scala> def transpose(x: List[List[Int]]): List[List[Int]] = {
     |   val b = new ListBuffer[List[Int]]
     |   var y = x filter (!_.isEmpty)
     |   while (!y.isEmpty) {
     |     b += y map (_.head)
     |     y = y map (_.tail) filter (!_.isEmpty)
     |   }
     |   b.toList
     | }

edited Dec 07 '12 at 16:59

answered Nov 06 '09 at 00:47

andrew cooke

45,717
10
93
143

I don't think this is odd functionality at all; I have definitely had cause to write functions which did exactly this – oxbow_lakes Nov 06 '09 at 11:30
I think Andrew meant that he's surprised that the standard library doesn't have such a function. – pr1001 Nov 06 '09 at 18:42
No, I really meant it seemed odd, because you seem to be losing some information (you can't reverse it to get what you started with). But I guess I can imagine uses if I try hard enough :) – andrew cooke Nov 08 '09 at 17:33
Oh, I see. I would say it'd be used in similar situations as zip if you have more than 2 lists. In my case, I have a list of lists of sensor readings. Transposing them gives me a list of lists of the readings at common times, which I can then reduce to the minimum, maximum, and average values. – pr1001 Nov 08 '09 at 17:48
Another useful variation is to pad out the short rows with Nothing, so operate on List[List[Option[A]]], make sure all lists are of the same length, that is, that you have a rectangular array of Option[A], then use library transpose. This is a different spec than your original, in which values "fall leftward" through empty slots, but it's a potentially useful variation. – Reb.Cabin Aug 24 '11 at 13:51
Warning: This is not a stack safe algorithm. You can only run it with a relatively small matrix. – Jiří Vypědřík Aug 20 '19 at 15:01

Rok Kralj · Answer 4 · 2017-01-02T02:02:36.753

This is probably the cleanest:

def transpose[T](l: List[List[T]]): List[List[T]] =
     l.flatMap(_.headOption) match {
         case Nil => Nil
         case head => head :: transpose(l.map(_.drop(1)))
     }

or a modified version that is even more efficient:

def transpose[T](l: List[List[T]]): List[List[T]] =
     l.flatMap(_.headOption) match {
         case Nil => Nil
         case head => head :: transpose(l.collect { case _ :: tail => tail })
     }

score 0 · Answer 5 · answered Jan 10 '17 at 11:24

0

How about this one-liner using the Scala's standard Api:

((l map (_.toArray)) toArray).transpose map (_.toList) toList

This gets the job done and is O(N*M), where N is the length of the wrapper list and M is the length of the longest list inside the wrapper list.

answered Jan 10 '17 at 11:24

GA1

1,568
2
19
30

1

Wow, thats a nice solution! – JoeDortman Jul 03 '18 at 15:44

Is there a safe way in Scala to transpose a List of unequal-length Lists?

5 Answers5

Linked