Simple function to read lines from a file in Scala lazily using scala-arm

Question

I want to write a Scala function to read all of the lines from a file lazily (i.e returning an Iterator[String]) which also closes the file afterwards. I know about the idiom io.Source.fromFile("something.txt").getLines however as noted here this will not close the file afterwards. Surely there is a simple way to do this?

Currently I'm using this, with the scala-arm library:

import resource.managed
import io.{Source, BufferedSource}
def lines(filename: String): Iterator[String] = {
  val reader = managed(Source.fromFile(filename, "UTF-8"))
  reader.map(_.getLines).toTraversable.toIterator
}

but this seems to read the whole file into memory as far as I can tell.

I come from a Python background, where this is laughably trivial:

def lines(filename):
   with open(filename) as f:
       for line in f:
           yield line

Surely there is a reasonably straightforward Scala equivalent which I just haven't managed to work out yet.

Why do you think it reads the entire file into memory? `getLines` returns an `Iterator[String]` — yan, Sep 25 '13 at 02:30
Yep, `getLines` is lazy, like I want, but something that `scala-arm` is doing when it calls it is reading everything into memory -- when I run `println(lines("BIGFILE.txt").slice(1, 10).toList)` I get an `OutOfMemoryError`, while `println(Source.fromFile("BIGFILE.txt").getLines.slice(1, 10).toList)` runs without problems (although doesn't close the file afterwards apparently). — Andy MacKinlay, Sep 25 '13 at 03:01

Simple function to read lines from a file in Scala lazily using scala-arm

0 Answers0