Deal with Java NIO Iterator in Scala with Try

Question

I recently learned how to use Scala's native Try type to handle errors. One good thing with Try is that I'm able to use for-comprehension and silently ignore the error.

However, this becomes a slight problem with Java's NIO package (which I really want to use).

val p = Paths.get("Some File Path")
for {
  stream <- Try(Files.newDirectoryStream(p))
  file:Path <- stream.iterator()
} yield file.getFileName

This would have been perfect. I intend to get all file names from a directory, and using a DirectoryStream[Path] is the best way because it scales really well. The NIO page says DirectoryStream has an iterator() method that returns an iterator. For Java's for loop, it's enough and can be used like this:

try (DirectoryStream<Path> stream = Files.newDirectoryStream(dir)) {
    for (Path file: stream) {
        System.out.println(file.getFileName());
    }
}

However, Scala does not accept this. I was greeted with the error:

[error] /.../DAL.scala:42: value filter is not a member of java.util.Iterator[java.nio.file.Path]
[error] file:Path <- stream.iterator

I try to use JavaConverters, and it shows it handles Java Iterator type: scala.collection.Iterator <=> java.util.Iterator, but when I try to call it in this way: stream.iterator().asScala, the method is not reachable.

What should I do? How do I write nice Scala code while still using NIO package?

score 2 · Accepted Answer · edited May 23 '17 at 12:02

I don't actually quite get while in this for comprehension filter is being invoked, but note that stream.iterator() returns a Iterator[Path], not a Path, even though my IDE thinks it does, probably because he thinks he can apply map to it, but in truth this are methods which are not defined on java.util.Iterator[java.nio.file.Path] as the compiler confirms:

scala>   for {
 |     stream <- Try(Files.newDirectoryStream(p))
 |     file <- stream.iterator()
 |   } yield file
<console>:13: error: value map is not a member of java.util.Iterator[java.nio.file.Path]
              file <- stream.iterator()

This for comprehension translates to:

Try(Files.newDirectoryStream(p)).flatMap(stream => stream.iterator().map(...))

Where the second map is not defined. One solution could be found in this SO question, but I can't tell you how to use iterator in for comprehension here since in java iterator cannot be mapped on and I'm not sure you can convert it into the comprehension.

Edit:

I managed to find out more about the problem, I tried this for comprehension:

for {
  stream <- Try(Files.newDirectoryStream(p))
  file <- stream.iterator().toIterator
} yield file

This doesn't compile because:

found   : Iterator[java.nio.file.Path]
required: scala.util.Try[?]
       file <- stream.iterator().toIterator

It translates to:

Try(Files.newDirectoryStream(p)).flatMap(stream => stream.iterator().map(...))

But flatMap actually expects a Try back, in fact this works:

Try(Files.newDirectoryStream(p)).flatMap(stream => Try(stream.iterator().map(...)))
                                                    ^

What I came up with:

import java.nio.file.{Paths, Files}
import util.Try
import scala.collection.JavaConversions._

Try(Files.newDirectoryStream(p))
  .map(stream => 
    stream
    .iterator()
    .toIterator
    .toList
    .map(path => path.getFileName.toString)
).getOrElse(List())

Which returns a List[String], unfortunately this is far from being as pretty as your for comprehension, maybe somebody else has a better idea.

Thank you for that answer, it looks quite fantastic. The transformation however takes time, and especially when the stream is quite huge. The original reason for Java Stream is used in NIO is to scale...but nonetheless, great great answer! — windweller, Aug 07 '14 at 00:49
Emm, unfortunately it seems like I don't even have `toIterator` method. How did you get it??? — windweller, Aug 07 '14 at 01:12
Yeah..I did import that one, but it didn't work. That's weird. But Thank you for helping out! In the pursuit of efficiency I wrote a slightly different function :) — windweller, Aug 09 '14 at 18:06

score 0 · Answer 2 · answered Aug 09 '14 at 18:07

I really like what Ende Neu wrote and it's hard to work with NIO in Scala. I want to preserve the efficiency brought from Java's Stream, so I decide to write this function instead. It still uses Try and I only need to deal with Success and Failure cases :)

It's not as smooth as I'd hope, and without Java 7's great try-with-resource feature, I have to close the stream by myself (which is terrible...), but this works out.

  def readFileNames(filePath: String):Option[List[Path]] = {
    val p = Paths.get(filePath)
    val stream: Try[DirectoryStream[Path]] = Try(Files.newDirectoryStream(p))

    val listOfFiles = List[Path]()
    stream match {
      case Success(st) =>
        val iterator = st.iterator()
        while (iterator.hasNext) {
          listOfFiles :+ iterator.next()
        }
      case Failure(ex) => println(s"The file path is incorrect: ${ex.getMessage}")
    }
    stream.map(ds => ds.close())
    if(listOfFiles.isEmpty) None else Some(listOfFiles)
  }

Deal with Java NIO Iterator in Scala with Try

2 Answers2