2

I've faced an interesting thing today regarding RandomAccessFile.

I've noticed that using RandomAccessFile's writeInt(int i) method is much more slower than using RandomAccessFile's write(byte[] b) where I first convert int value to byte[4] array.

I'm doing the conversion with this code

private static byte[] intToByte(int i)
{
   byte[] result = new byte[4];

   result[0] = (byte) (i >> 24);
   result[1] = (byte) (i >> 16);
   result[2] = (byte) (i >> 8);
   result[3] = (byte) (i);

  return result;
}

The difference is very significant, favoring write(byte[] b).

Writing 1 million ints on my laptop with JDK 8:

  • via writeInt(int i) method took ~9 seconds
  • via write(byte[] b) took ~2,3 seconds

I have similar results in another environment, where I'm using JDK 7 and a totally different machine.

The writeInt(int i) method delegate to native write0(int b) method and write(byte[] b) delegates to native writeBytes.

When I did profiling I've noticed that the majority of the execution time was spent in writeInt method when it was used.

Does anyone know why I see such a big difference? Seems like writeInt is way less efficient.

Kristoff
  • 326
  • 2
  • 13
  • 1
    how about calling those methods in reverse? you might have heated the JVM with the first one, only to call the second with a hot VM. anyway there is a reason JMH exists and is praised so much around here; unless you have really found something interesting ;) – Eugene Feb 05 '18 at 20:18
  • Thanks for the comment. I understand what you are saying, I will do the JMH benchmark and get back with the results. Although I was doing both tests on a "cold" JVM, calling those tests separately in a different runs - same results. – Kristoff Feb 05 '18 at 21:12
  • understood, but at the same time you need to make a clear distinction between some numbers and some numbers that make sense. With a proper JMH test (this is ain't going to be a walk in the park either), your numbers might make sense... – Eugene Feb 05 '18 at 21:18
  • Ok, I have some numbers. Test source can be found here https://github.com/kristoffSC/RafBenchmark/tree/master/src/main/java/org/home/benchmark/jmh Seems that write(byte[] b) is faster even in JMH. Results can be found here: https://github.com/kristoffSC/RafBenchmark/blob/master/JMH_Results.txt One thing though - because the raf.writeInt and raf.write(byte[] b) methods are not returning any value I was not able to use JMH blackhole here. – Kristoff Feb 05 '18 at 22:29
  • If you're not seeking in between writes, just writing sequentially, you shouldn't be using `RandomAccessFile` at all. A `DataOutputStream`around a `BufferedOutputStream` around a `FileOutputStream` will be orders of magnitude faster. RAF is for, err, random access files. It isn't optimised in any way. – user207421 Feb 05 '18 at 23:00
  • I appreciate the comment @EJP but this is not about "how to writ to file as fast as you can". I'm asking here about performance difference between two RAF's methods. – Kristoff Feb 06 '18 at 07:22
  • I do not understand. If you aren't concerned about performance, why are you concerned about performance? – user207421 Feb 06 '18 at 09:13
  • No worries, I this case - I'm only interested why this works like it works :) BTW regarding the File read performance in general - https://mechanical-sympathy.blogspot.com/2011/12/java-sequential-io-performance.html – Kristoff Feb 06 '18 at 09:18
  • I don't know why you are citing an article about Java I/O sequential performance when you aren't doing sequential I/O, and when I have already pointed out that if you were, with buffering, it would all be thousands of times quicker. – user207421 Feb 06 '18 at 09:23

2 Answers2

2

RandomAccessFile has actually two native methods to write bytes:

//writes an array
private native void writeBytes(byte b[], int off, int len) throws IOException;

and

//writes one byte
public native void write(int b) throws IOException;

the method writeInt(int) writes each byte separately with the native write(int) method, while write(byte[]) uses the native writeBytes(byte[],int,int) method.

the writeInt method does 4 method invocations to write each byte of the passed integer value, the other method uses only one invocation to write the array. Method invocations are actually expensive operations in java: for each invocation the JVM allocates additional memory for the operand stack and the local variables array.

neoexpert
  • 465
  • 1
  • 10
  • 20
1

Not going to go into details of the changes that I made, but your tests are a little bit flawed. I took the liberty of updating them a little and ran a few tests too:

@BenchmarkMode(value = { Mode.AverageTime })
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@Warmup(iterations = 2, time = 2, timeUnit = TimeUnit.SECONDS)
@Measurement(iterations = 2, time = 2, timeUnit = TimeUnit.SECONDS)
public class RandomAccessWriteFileTest {

    public static void main(String[] args) throws Exception {
        Options opt = new OptionsBuilder().include(RandomAccessWriteFileTest.class.getSimpleName())
                .jvmArgs("-ea")
                .shouldFailOnError(true)
                .build();
        new Runner(opt).run();
    }

    @Benchmark()
    @Fork(1)
    public long benchamrkWriteDirectInt(BenchmarkPlainIntSetup setupTest) {
        try {
            setupTest.raf.writeInt(6969);
            return setupTest.raf.length();
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }

    @Benchmark()
    @Fork(1)
    public long benchamrkWriteConvertedInt(BenchmarkConvertedIntSetup setupTest) {
        try {
            setupTest.raf.write(intToBytes(6969));
            return setupTest.raf.length();
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }

    private static byte[] intToBytes(int i) {
        byte[] result = new byte[4];

        result[0] = (byte) (i >> 24);
        result[1] = (byte) (i >> 16);
        result[2] = (byte) (i >> 8);
        result[3] = (byte) i;

        return result;
    }

    @State(Scope.Thread)
    static public class BenchmarkConvertedIntSetup {

        public RandomAccessFile raf;

        public File f;

        @Setup(Level.Iteration)
        public void setUp() {
            try {
                f = new File("jmhDirectIntBenchamrk.ser" + ThreadLocalRandom.current().nextInt());
                raf = new RandomAccessFile(f, "rw");
            } catch (FileNotFoundException e) {
                throw new RuntimeException(e);
            }
        }

        @TearDown(Level.Iteration)
        public void tearDown() {
            f.delete();
        }
    }

    @State(Scope.Thread)
    static public class BenchmarkPlainIntSetup {

        public RandomAccessFile raf;

        public File f;

        @Setup(Level.Iteration)
        public void setUp() {
            try {
                f = new File("jmhDirectIntBenchamrk.ser" + ThreadLocalRandom.current().nextInt());
                raf = new RandomAccessFile(f, "rw");
            } catch (FileNotFoundException e) {
                throw new RuntimeException(e);
            }
        }

        @TearDown(Level.Iteration)
        public void tearDown() {
            f.delete();
        }
    }
}

Absolutely there is a difference in results (these are ms per operation)

 benchamrkWriteConvertedInt  0.008 
 benchamrkWriteDirectInt     0.026

No idea why (may be will dig the assembly to understand some time later, but I can confirm your findings. good question!)

This was run with latest java-8 and java-9 btw

Eugene
  • 117,005
  • 15
  • 201
  • 306
  • Thanks for this! Could you create a Pull Request with your changes? – Kristoff Feb 06 '18 at 09:29
  • @Kristoff np, but I am way too busy right now :( sorry – Eugene Feb 06 '18 at 09:37
  • Its, cool - I dont need PR any more, just took the code from your answer. I have Similar results for after your changes RandomAccessWriteFileTest.benchamrkWriteConvertedInt 0,004 ms/op RandomAccessWriteFileTest.benchamrkWriteDirectInt 0,011 ms/op – Kristoff Feb 06 '18 at 19:41
  • @Kristoff well the next step would be to actually look at the disassembly code or better try to understand the native implementations... again, *I hope* to have some time today to try to do that (because boy, this is intriguing!) :) – Eugene Feb 06 '18 at 19:43
  • There is overhead everytime you make a call to write something to disk. So when writing to disk, it is better to batch up your writes into fewer calls. In the convertedInt method this is what you are doing. You would likely get even better results if you wrote bigger chunks at a time. – Jay Askren Jul 03 '18 at 23:05