1

I want to read a 30GB file using MappedByteBuffer.

try (FileChannel sourceChannel = new RandomAccessFile("sample.csv", "r").getChannel()) {
    MappedByteBuffer buf = sourceChannel.map(FileChannel.MapMode.READ_ONLY, 0, sourceChannel.size());
}

Problem:

java.lang.IllegalArgumentException: Size exceeds Integer.MAX_VALUE
    at java.base/sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:941)

What could I do here?

membersound
  • 81,582
  • 193
  • 585
  • 1,120
  • 2
    Not try to memory map files larger than 2GB. – Kayaman Oct 22 '19 at 11:19
  • So can I memory-map the files partially and repeat? My goal is to perform a bytewise copy of the file, but I want to evaluate the content and remove some parts conditionally. Preferably by not having to read each line fully as `String`, as any string line will have to be converted back into bytes during writing... – membersound Oct 22 '19 at 11:20
  • 2
    You're trying to solve your earlier problem, but this tool won't help you. You're not doing random access, you're doing ETL. The fastest way to do it is with `BufferedReader/BufferedWriter`. – Kayaman Oct 22 '19 at 11:23

0 Answers0