3

I in the process of writting a simple java program which reads the contents of the directory and print out the names of the files and last modified time .

The issue i forsee is , the vault i am reading is pretty huge and there are some case where the files in a single directory can well exceed 20000. Using file api 

    `file.listFiles()`

would inturn create 20000 file objects, my concern this could slow down the process , may be bloat the memory as well.

Is there a way to batch i.e to tell java to scan the directory in terms of 50 files at a time or atleast iterate one file at a time instead of loading all objects in memory at once

Sudhakar
  • 4,823
  • 2
  • 35
  • 42

2 Answers2

5

You should use a java.nio.file.FileVisitor via java.nio.file.Files.walkFileTree(...). It has been introduced in Java 7 exactly for this use case.

Jerome
  • 8,427
  • 2
  • 32
  • 41
1

I wouldn't worry about the memory, I would worry about the disk access time for those files.

The access time for a file can be 8 ms, even for a SSD it can be 100 micro-seconds. The time it takes to create a File object can be less than a micro-second.

You are right that 20,000 File objects is non trivial, but if this your main problem you are doing very well.

You can use list() instead which will be slightly more memory efficient and create File only as you need them.

Peter Lawrey
  • 525,659
  • 79
  • 751
  • 1,130
  • But then to access the last modified timestamp , i would have to create a File Object with the file Name , that can only slow the process ..dont you think – Sudhakar Apr 29 '12 at 15:37
  • It would make a difference of a few micro-seconds, far less than a single file lastModified() access. I would write the code in the simplest way you can. – Peter Lawrey Apr 29 '12 at 15:44