Creating Large Files in Java (2gb+)

Question

I'm currently working on a project to create a large random binary file (2gb to 10gb) and I'm at a loss for ideas. Obviously writing out 10gb worth of data would take forever and I need a timely approach. My latest idea was to create a random access file and set the size to 2+ gb but it seems that the .setLength function can't handle sizes that large. I've also thought about just appending one file to another to get a file large enough but it seems like there is no really good way to do that quickly either. Any ideas out there?

It's going to take time, just write it without running out of memory. You can't do multi-threading for file creation. — Bhesh Gurung, Nov 09 '12 at 19:29
what are your needs in the project ? how many files are required to be generated ? and what is maximum acceptable file generating time for your needs ? — , Nov 09 '12 at 20:11
The `setLength` function of RandomAccessFile accepts a long. There is no file size it cannot create now that doesn't exceed most, if not all existing storage limits. What problem are you having using it? — Perception, Nov 09 '12 at 20:19
I just need one file, and I was attempting to send it 2147483648 for 2gb and it was throwing an error but for 1073741824 for 1 gb it handled it fine. I would like the file to generate in under a minute — Dommol, Nov 09 '12 at 23:09
Figured out my problem. Thanks to Perception I noticed that I was sending the .setLength() method an int instead of a long and the error I was getting on values above 1GB was because it was switching it to a negative number due to the size constraints on integers. Thanks for all the help, the program is up and running — Dommol, Nov 10 '12 at 05:20

score 0 · Answer 1 · answered May 12 '22 at 21:18

This question and the comments helped me a lot recently - to create a file of a given size, this would be a method one could write:

 private File createFileOfSize(String suffix, long requestedSize) throws IOException {
    File tempFile = new File(temporaryDirectory, getClass().getSimpleName() + "." + suffix);
    RandomAccessFile randomAccessFile = new RandomAccessFile(tempFile, "rw");
    randomAccessFile.setLength(requestedSize);
    randomAccessFile.close();
    return tempFile;
  }

... where RandomAccessFile is java.io.RandomAccessFile

Creating Large Files in Java (2gb+)

1 Answers1