4

I have folder with two million of images 1-2KB. I try to read

for (File f : tmpDir.listFiles()) {
   long t1 = System.currentTimeMillis();
   image = ImageIO.read(f);
   long time = System.currentTimeMillis() - t1;

}

first 20,000 files time = 0 or time = 1;

then time of reading increased up to 50-100 mls. (time = 50-1000)

How to improve performance?

user1367713
  • 286
  • 2
  • 4
  • 11
  • on 64 linux, ext4 filesystem – user1367713 Sep 15 '15 at 16:28
  • 2
    Are you sure this is actually the IO, vs garbage collection for all those images you've created, for example? What happens if you just read each file, ignoring the actual data (thus avoiding repeated allocation)? – Jon Skeet Sep 15 '15 at 16:39
  • Did you try to increase max heap space to reduce garbage collection? – Puce Sep 15 '15 at 16:42
  • No, it is not a GC problem. – user1367713 Sep 16 '15 at 07:04
  • I agree with Jon Skeet, it looks like a Java GC issue caused by the many object allocations/deallocations. Here are a couple of ideas to prove this: (1) have a single file in the directory and read it over and over again -- I expect you will get the same results; (2) increase the java heap size -- I expect to see the latency increase much later; (3) monitor I/O traffic -- I expect you will find the block device underutilized (this means it is not an I/O issue); (4) monitor CPU usage -- I expect you will see a high userspace CPU utilization and very small kernel/system CPU utilization. – Radu Oct 16 '15 at 12:04

0 Answers0