consume items from a scala Iterator

Question

I'm confused about behavior of method take in trait Iterator. It seems that it doesn't consume items. Here is an example:

scala> Iterator(1,2,3)
res0: Iterator[Int] = non-empty iterator

scala> res0 take 2 toArray
res1: Array[Int] = Array(1, 2)

scala> res0.next
res2: Int = 1

Apparently step 2 consumes two items, but in step 3 the Iterator is still at first item. Looking at the implementation, I can't see any kind of copying or buffering, just a new Iterator which delegates to the underlying one. How could it be possible? How can I manage to really consume n items?

retronym · Accepted Answer · 2011-10-01T12:03:31.723

The iterator in question is defined in IndexedSeqLike#Elements (source). A ticket was recently filed about the the inconsistent behaviour of take across different iterator implementations.

To really consume N items, call Iterator#next N times.

You might want to consider using Stream, which is a lazy (like Iterator), but is also immutable (unlike Iterator).

scala> val s = Stream(1, 2, 3)
s: scala.collection.immutable.Stream[Int] = Stream(1, ?)

scala> s.take(2).toList
res43: List[Int] = List(1, 2)

scala> s.take(2).toList
res44: List[Int] = List(1, 2)

scala> s.drop(2).toList
res45: List[Int] = List(3)

scala> {val (s1, s2) = s.splitAt(2); (s1.toList, s2.toList)}
res46: (List[Int], List[Int]) = (List(1, 2),List(3))

Thanks retronym, I see the point. What made me discard `Stream` was the lack of something like a "current item pointer" (by the way, maybe due to Java influence, something that one might expect in a so called thing). Now I see that I could get a "current pointer" with a sequence of `drop` and look at `head`. I will try again. — jglatre, Oct 02 '11 at 09:29

score 2 · Answer 2 · answered Oct 02 '11 at 12:07

2

Thanks guys.

This is my solution to consume bunches of items from an Iterator:

  implicit def consumable(i: Iterator[_]) = new {
    def next(n: Int) = {
      (for (_ <- 1 to n) yield i.next()).iterator
    }
    def skip(n: Int) {
      (1 to n).foreach(_ => i.next())
    }
  }

any comments will be welcome.

answered Oct 02 '11 at 12:07

jglatre

911
1
12
15

To be consistent with return type the definition should be: implicit def consumable[T](i: Iterator[T]) otherwise it returns Iterator[Any] – Miquel Feb 13 '15 at 11:09

score 1 · Answer 3 · edited Sep 18 '13 at 08:00

1

You want to consume the items, drop them. Note that most methods called on an Iterator will make that Iterator useless for further use -- useless in the sense that behavior is undefined and subject to change.

edited Sep 18 '13 at 08:00

ndeverge

21,378
4
56
85

answered Oct 01 '11 at 15:13

Daniel C. Sobral

295,120
86
501
681

`drop` is actually subject to the same inconsistency as `take`. `{val i = Iterator(1, 2, 3); i.drop(1); i.next}` returns `1`, not `2`. – retronym Oct 01 '11 at 16:07
1

@retronym Once you called `drop` on `i`, `i` cannot be reliably used anymore. Instead, you have to use what `drop` returned. – Daniel C. Sobral Oct 01 '11 at 19:09
Good point. `{val i = Iterator(1, 2, 3); i.drop(1).next}` gives 2. Everything but `next()` and `hasNext()` (potentially) invalidates the original iterator; although those methods may return a *new*, valid, iterator. I look forward to your documentation contribution to clarify this. – retronym Oct 01 '11 at 20:32

consume items from a scala Iterator

3 Answers3