Java NIO Read Folder's content's attributes at once

Question

I'm writing a backup program using Java and the package NIO.

As far as I found, I could do it as in the code example below, i.e. read the folder content list and then for each file I have to do file attributes request... this is not an effective approach, especially if someone has big folders with thousands of files there in one folder, so maybe there is another way to do it by reading folder's content with all its attributes?

Or maybe should I use something else instead of NIO for this case? Thank You very much

public void scan(Path folder) throws IOException {
    try (DirectoryStream<Path> ds = Files.newDirectoryStream(folder)) {
        for (Path path : ds) {
            //Map<String, Object> attributes = Files.readAttributes(path, "size,lastModifiedTime");
        }
    }
}

There is no other way. Even operating system level APIs have no alternative to this approach. What would be the return type of a method capable of “reading folder's content with all its attributes”? — VGR, Jun 16 '21 at 15:40
This is a question how does it work on the lower level... for example, either in Windows and Linux if I type "dir" or "ls -la" - then I will get all the required info... so I thought that maybe when Java reads the content of the folder, on the lower level it also reads the attributes and then just omit it... Could You confirm or refute this please? — Sunny, Jun 16 '21 at 15:46
Those commands do the same thing internally: they retrieve the attributes for each file, one by one. [The source of ls](https://github.com/wertarbyte/coreutils/blob/master/src/ls.c) shows that it uses readdir(3), which returns one entry’s attributes at a time. — VGR, Jun 16 '21 at 15:58
This may help answer your question, using Files.find and you can use the attributes in the predicate and also inside the stream without calling Files.readAttributes yourself. [How to get Streams of File Attributes from the FileSystem?](https://stackoverflow.com/questions/66699379/how-to-get-streams-of-file-attributes-from-the-filesystem) — DuncG, Jun 16 '21 at 16:18
@DuncG thank You very much, by briefly looking it seems that Your link is exactly what I need, but I need to test it first... So I hope, it will work for me :) I will add the result of my benchmarking on few gigabytes of different scrap files... — Sunny, Jun 16 '21 at 18:38

score 0 · Accepted Answer · answered Jul 15 '21 at 15:04

Thanks to DuncG, the answer is very simple:

HashMap <Path, BasicFileAttributes> attrs = new HashMap<>();
BiPredicate<Path, BasicFileAttributes> predicate = (p, a) -> {
    return attrs.put(p, a) == null;
};
Stream<Path> stream = Files.find(folder, Integer.MAX_VALUE, predicate);

I made a benchmark to compare, this example executes 3x times faster than the example from the question text, so it seems that it invokes fewer filesystem I/O operations... in theory...

Java NIO Read Folder's content's attributes at once

1 Answers1