Why is Seq.newBuilder returning a ListBuffer?

Question

Looking at

val sb = Seq.newBuilder[Int]
println(sb.getClass.getName)
sb += 1
sb += 2
val s = sb.result()
println(s.getClass.getName)

the output is

scala.collection.mutable.ListBuffer
scala.collection.immutable.$colon$colon

using Scala 2.10.1.

I would expect Seq.newBuilder to return a VectorBuilder for example. This is returned by CanBuildFrom, if the result is explicitly typed to a Seq:

def build[T, C <: Iterable[T]](x: T, y: T)
                              (implicit cbf: CanBuildFrom[Nothing, T, C]): C = {
  val b = cbf()
  println(b.getClass.getName)
  b += x
  b += y
  b.result()
}

val s: Seq[Int] = build(1, 2)
println(s.getClass.getName) // scala.collection.immutable.Vector

in this case the builder is a VectorBuilder, and the result's class is a Vector.

So I explicitly wanted to build a Seq, but the result is a List which needs more RAM, according to Scala collection memory footprint characteristics.

So why does Seq.newBuilder return a ListBuffer which gives a List in the end?

Kind of a dupe of your http://stackoverflow.com/q/17599652/1296806, but I really didn't see what you had meant by your remark about CanBuildFrom. That's pretty funny. As an aside, I think maybe a question is a dupe if it elicits the same answers. Don't forget to read the overview doc and that Spiewak http://stackoverflow.com/a/6934116/1296806 — som-snytt, Jul 20 '13 at 00:34
@som-snytt If a `List` is the default implementation for `Seq`, then I would expect `CanBuildFrom` to return a `List` as well, if the result is typed to `Seq` (for consistency). I can add this comment now, because one of the answers stated the fact that `List` is the default for `Seq`. As for Spiewak's answer, I have read it before, but this raised yet another question: Both `Seq` and `List` are traits whereas `Vector` is not. But all I want is a `Seq` (`Iterable` with order), for example as the result of a method. The most efficient impl. should be used to fulfill the contract. — Beryllium, Jul 20 '13 at 07:56
The notion of "most efficient" is contextual, so people argue about it. I think historically, Lisp types think in terms of List. However,for CanBuildFrom, it clearly matters what you're building from, hence CanBuild *From*, so you're not off the hook there. But I see what you mean: you'd like to ask for a default CBF. Well, the default you get happens to be manufactured for a common use case, String, where you clearly want an indexed Seq. — som-snytt, Jul 20 '13 at 08:34
I can't believe SO applies Twitterish length restrictions on comments. Anyway: Why? I don't know. You could argue that Strings should be processed left to right; except that people assume string is backed by something arraylike. We do s.indexOf(c) all the time. — som-snytt, Jul 20 '13 at 08:34
@som-snytt The context is the `Seq` trait. So I'd like to get the "most efficient" implementation for that. — Beryllium, Jul 20 '13 at 13:23

Fynn · Accepted Answer · 2013-07-19T15:45:03.723

The Scala Collections API is very complex and its hierarchy is rich in depth. Each level represents some sort of new abstraction. The Seq trait split up into two different subtraits, which give different guarantees for performance (ref.):

An IndexedSeq provides fast random-access of elements and a fast length operation. One representative of this IndexedSeq is the Vector.
A LinearSeq provides fast access only to the first element via head, but also has a fast tail operation. One representative of this LinearSeq is the List.

As the current default implementation of a Seq is a List, Seq.newBuilder will return a ListBuffer. However, if you want to use a Vector you can either use Vector.newBuilder[T] or IndexedSeq.newBuilder[T]:

scala> scala.collection.immutable.IndexedSeq.newBuilder[Int]
res0: scala.collection.mutable.Builder[Int,scala.collection.immutable.IndexedSeq[Int]] = scala.collection.immutable.VectorBuilder@1fb10a9f

scala> scala.collection.immutable.Vector.newBuilder[Int]
res1: scala.collection.mutable.Builder[Int,scala.collection.immutable.Vector[Int]] = scala.collection.immutable.VectorBuilder@3efe9969

I was aware of the two sub traits, abut in fact I did not know that it is just that simple: The default implementation for a `Seq` is a `List`. I have just not seen that piece of information, because I looked at it only from the perspective of the `Seq` *trait*, but of course, yes, next time I'll look at the *companion* as well. — Beryllium, Jul 20 '13 at 13:02

score 4 · Answer 2 · answered Jul 19 '13 at 15:49

4

The default Seq implementation is List:

Seq(1, 2, 3)  // -> List(1, 2, 3)

...thus ListBuffer is the correct builder. If you want Vector, use Vector.newBuilder or IndexedSeq.newBuilder.

answered Jul 19 '13 at 15:49

0__

66,707
21
171
266

Yes, it's just that simple: It's clearly stated in the doc for the `Seq` *companion*, and I have only read the doc for the `Seq` trait. – Beryllium Jul 20 '13 at 13:04

som-snytt · Answer 3 · 2013-07-20T19:01:46.813

OK, but you're not going to believe it. Turning on -Yinfer-debug for your CanBuildFrom counter-example,

[search] $line14.$read.$iw.$iw.build[scala.this.Int, Seq[scala.this.Int]](1, 2) with pt=generic.this.CanBuildFrom[scala.this.Nothing,scala.this.Int,Seq[scala.this.Int]] in module class $iw, eligible:
  fallbackStringCanBuildFrom: [T]=> generic.this.CanBuildFrom[String,T,immutable.this.IndexedSeq[T]]
[solve types] solving for T in ?T
inferExprInstance {
  tree      scala.this.Predef.fallbackStringCanBuildFrom[T]
  tree.tpe  generic.this.CanBuildFrom[String,T,immutable.this.IndexedSeq[T]]
  tparams   type T
  pt        generic.this.CanBuildFrom[scala.this.Nothing,scala.this.Int,Seq[scala.this.Int]]
  targs     scala.this.Int
  tvars     =?scala.this.Int
}
[search] considering no tparams (pt contains no tvars) trying generic.this.CanBuildFrom[String,scala.this.Int,immutable.this.IndexedSeq[scala.this.Int]] against pt=generic.this.CanBuildFrom[scala.this.Nothing,scala.this.Int,Seq[scala.this.Int]]
[success] found SearchResult(scala.this.Predef.fallbackStringCanBuildFrom[scala.this.Int], ) for pt generic.this.CanBuildFrom[scala.this.Nothing,scala.this.Int,Seq[scala.this.Int]]
[infer implicit] inferred SearchResult(scala.this.Predef.fallbackStringCanBuildFrom[scala.this.Int], )

and indeed,

  implicit def fallbackStringCanBuildFrom[T]: CanBuildFrom[String, T, immutable.IndexedSeq[T]] =
    new CanBuildFrom[String, T, immutable.IndexedSeq[T]] {
      def apply(from: String) = immutable.IndexedSeq.newBuilder[T]
      def apply() = immutable.IndexedSeq.newBuilder[T]
    }

What do you mean, your Iterable is not a String?

trait CanBuildFrom[-From, -Elem, +To]

Such is the evil of inferring either Nothing or Any.

Edit: Sorry, I misspoke, I see that you told it Nothing explicitly.

Update:

Since CBF is contravariant in From, a CBF from String serves as a CBF from Nothing.

scala> typeOf[CanBuildFrom[Nothing,Int,Seq[Int]]] <:< typeOf[CanBuildFrom[String,Int,Seq[Int]]]
res0: Boolean = false

scala> typeOf[CanBuildFrom[String,Int,Seq[Int]]] <:< typeOf[CanBuildFrom[Nothing,Int,Seq[Int]]]
res1: Boolean = true

For instance, if you need to build from an immutable.Map, you'd want a CBF from collection.Map to work.

As someone else commented, it's just weird for Nothing. But you get what you asked for. That is, you underspecified, which means you don't mind much what you get back, Vector or whatever.

I do appreciate your effort, but I just fail to understand what you trying to show. Does it say that the resulting `IndexedSeq` is created by a `fallbackStringCanBuildFrom`? BTW I did learn about `-Yinfer-debug` to shine some light on the magic around CBF. — Beryllium, Jul 20 '13 at 13:50
@Beryllium Yes! That might be obvious to some smarties, but not to me. — som-snytt, Jul 20 '13 at 18:52

score 0 · Answer 4 · answered Jul 19 '13 at 15:33

I agree that this is weird. Why don't you just use Vector.newBuilder, if that's what you're looking for?

scala> val sb = Vector.newBuilder[Int]
sb: scala.collection.mutable.Builder[Int,scala.collection.immutable.Vector[Int]] = scala.collection.immutable.VectorBuilder@1fb7482a

scala> println(sb.getClass.getName)
scala.collection.immutable.VectorBuilder

scala> sb += 1
res1: sb.type = scala.collection.immutable.VectorBuilder@1fb7482a

scala> sb += 2
res2: sb.type = scala.collection.immutable.VectorBuilder@1fb7482a

scala> val s = sb.result()
s: scala.collection.immutable.Vector[Int] = Vector(1, 2)

scala> println(s.getClass.getName)
scala.collection.immutable.Vector

I am just looking for a `Seq` (`Iterable` with order), nothing more, nothing less. That's why I'd like to delegate the task of picking the best `Builder` and implementation class to the companion object generally speaking. Probably my question was not clear enough, the `VectorBuilder` was just an example of a possible alternative. — Beryllium, Jul 20 '13 at 13:19

Why is Seq.newBuilder returning a ListBuffer?

4 Answers4

Linked