Why does my stream copying procedure steadily degrades in performance

Question

This code works slower and slower during copying large files. Am I doing something wrong?

    InputStream ms2 = new BufferedInputStream(new FileInputStream("/home/fedd/Videos/homevid.mp4"));
    OutputStream fos2 = new BufferedOutputStream(new FileOutputStream("testfile2.mp4", true));

    try {
        int byt;
        int i = 0;
        long time = System.currentTimeMillis();
        while ((byt = ms2.read()) != -1) {
            fos2.write(byt);
            i++;
            if (i > 100000) {
                i = 0;
                long took = System.currentTimeMillis() - time;
                System.out.println("100000 bytes took " + took + " milliseconds which means " + (100000000 / took) + " bytes per second");
            }
        }
        fos2.close();
        ms2.close();
    } catch (Exception e) {
        throw new RuntimeException(e);
    }

My Java is:

openjdk 10.0.2 2018-07-17 OpenJDK Runtime Environment (build 10.0.2+13-Ubuntu-1ubuntu0.18.04.4)

OpenJDK 64-Bit Server VM (build 10.0.2+13-Ubuntu-1ubuntu0.18.04.4, mixed mode)

What is ms2 (ms2.read())? Check your while statement, I think there is a typo. — Ekamjit Singh, Mar 19 '19 at 08:02
You are measuring time from the very beginning of the loop. You need to reset begin timestamp after printing: `time = System.currentTimeMillis();`. — gudok, Mar 19 '19 at 08:02
BTW: it's pretty normal in most disks/SSD to have performance significantly decrease after a bit. That's due to write caches either at the kernel level or even at the hardware level. So I'd also simply test how fast you can copy the file using other tools... — Giacomo Alzetta, Mar 19 '19 at 08:03
You should use the try with resources statement (see https://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html ) or close the streams in the finally block else it introduces possible memory leaks due to open streams. — BeWu, Mar 19 '19 at 08:12
@EkamjitSingh, you're right, it was my InputStream. Wrong copy-paste — fedd, Mar 19 '19 at 11:15
final code =) https://github.com/vsetec/vsetec-utilities/blob/d23eea57da17fbbc4db2a892bbe734320548fbc3/vsetec-utilities/src/main/java/com/vsetec/utilities/ParallelStream.java#L215 — fedd, Mar 19 '19 at 11:16
@BeWu, thanks, will eventually do. This is for just a test, and I've never managed to sit until the end of the stream... — fedd, Mar 19 '19 at 11:19

score 6 · Accepted Answer · answered Mar 19 '19 at 08:07

You need to reset your base 'time' after each comparison. Try using this:

if (i > 100000) {
    i = 0;
    long took = System.currentTimeMillis() - time;
    time = System.currentTimeMillis();
    System.out.println("100000 bytes took " + took + " milliseconds which means " + (100000000 / took) + " bytes per second");
}

Tamas Hegedus · Answer 2 · 2019-03-19T15:19:20.133

5

Your performance degrades because your calculation is wrong. For the second chunk, you are calculating the bytes per second from the second chunks size, but from both chunks time. Try adding time = System.currentTimeMillis(); after long took = ...

edited Mar 19 '19 at 15:19

answered Mar 19 '19 at 08:04

Tamas Hegedus

28,755
12
63
97

You mean `time`, not `took`, but hell yes – fedd Mar 19 '19 at 08:15

Why does my stream copying procedure steadily degrades in performance

2 Answers2