15

I got a kind of weird question to ask - how can I break a given file into small pieces using Java (any file type)? Then I could put those pieces onto a number of CD's and/or pendrives and take them away. I tried doing it this way

But as most users commented, what I am trying is not achievable in that way. So, I decided to ask a new question for getting a correct method for breaking files.

When I break a file into pieces (say 30 pieces), there must be a way to reassemble them and recreate the original file. Please help.

kc2001
  • 5,008
  • 4
  • 51
  • 92
PeakGen
  • 21,894
  • 86
  • 261
  • 463

2 Answers2

45

Read portions of bytes to byte array and store them in new files when buffer is full or it is end of file.

For example (code is not perfect, but it should help understanding the process)

class FileSplit {
    public static void splitFile(File f) throws IOException {
        int partCounter = 1;//I like to name parts from 001, 002, 003, ...
                            //you can change it to 0 if you want 000, 001, ...

        int sizeOfFiles = 1024 * 1024;// 1MB
        byte[] buffer = new byte[sizeOfFiles];

        String fileName = f.getName();

        //try-with-resources to ensure closing stream
        try (FileInputStream fis = new FileInputStream(f);
             BufferedInputStream bis = new BufferedInputStream(fis)) {

            int bytesAmount = 0;
            while ((bytesAmount = bis.read(buffer)) > 0) {
                //write each chunk of data into separate file with different number in name
                String filePartName = String.format("%s.%03d", fileName, partCounter++);
                File newFile = new File(f.getParent(), filePartName);
                try (FileOutputStream out = new FileOutputStream(newFile)) {
                    out.write(buffer, 0, bytesAmount);
                }
            }
        }
    }

    public static void main(String[] args) throws IOException {
        splitFile(new File("D:\\destination\\myFile.mp4"));
    }
}

myFile.mp4 size=12,7 MB

After split I had 13 files

  • myFile.mp4.001 - myFile.mp4.012 with size 1 MB
  • myFile.mp4.013 with size 806 KB

If you want to merge these files you can use

public static void mergeFiles(List<File> files, File into)
        throws IOException {
    try (FileOutputStream fos = new FileOutputStream(into);
         BufferedOutputStream mergingStream = new BufferedOutputStream(fos)) {
        for (File f : files) {
            Files.copy(f.toPath(), mergingStream);
        }
    }
}

You can also create some additional methods to make your life easier. For instance method which will create list of files containing separated parts based on name (and location) of one of these files.

public static List<File> listOfFilesToMerge(File oneOfFiles) {
    String tmpName = oneOfFiles.getName();//{name}.{number}
    String destFileName = tmpName.substring(0, tmpName.lastIndexOf('.'));//remove .{number}
    File[] files = oneOfFiles.getParentFile().listFiles(
            (File dir, String name) -> name.matches(destFileName + "[.]\\d+"));
    Arrays.sort(files);//ensuring order 001, 002, ..., 010, ...
    return Arrays.asList(files);
}

With that method we can overload mergeFiles method to use only one of the files File oneOfFiles instead of whole list List<File> (we will generate that list based on one of the files)

public static void mergeFiles(File oneOfFiles, File into)
        throws IOException {
    mergeFiles(listOfFilesToMerge(oneOfFiles), into);
}

You can also overload these methods to use String instead of File (we will wrap each String in File when needed)

public static List<File> listOfFilesToMerge(String oneOfFiles) {
    return listOfFilesToMerge(new File(oneOfFiles));
}

public static void mergeFiles(String oneOfFiles, String into) throws IOException{
    mergeFiles(new File(oneOfFiles), new File(into));
}
Pshemo
  • 122,468
  • 25
  • 185
  • 269
  • And what about merge? – Umang Kothari Nov 27 '14 at 09:50
  • @pshemo Files.copy(f.toPath(), mergingStream); this line does not working Android, any alternative? – Umang Kothari Nov 28 '14 at 06:43
  • @ldce I used `Files` class from `java.nio.files` package. Java is opensource so you can see its code, for instance you can adapt [this method](http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b132/java/nio/file/Files.java#2899) to copy data from one stream to another. This way you can use it like `copy(new FileInputStream(sourceFile), mergingStream)`. – Pshemo Nov 28 '14 at 15:11
  • the code segment is not complete yet, should add `finally` and close streams. – Logan Guo Jan 22 '16 at 08:16
  • 2
    @LoganGuo Not quite (unless I misunderstood your comment). We don't need to handle any exception, even if there is `try` section. I used ***try-with-resources*** to ensure closing resource even if exception will be thrown (so no `finally` is needed), also I don't have `catch` sections there since how to handle exceptions is up to user which will use this method, which is why I am re-throwing them. – Pshemo Jan 22 '16 at 18:52
  • Cool! Great Work. –  Jun 20 '17 at 15:33
  • 1
    When handling large files, you could also consider using `java.nio.channels.FileChannel`, see https://stackoverflow.com/a/25548570/107013 – MyKey_ Jul 25 '17 at 09:58
  • @famfamfam Same way you would call them in `main` method, which is also a thread (a main thread of application). So since methods are static we call them using class name which declared them like `FileSplit.splitFile(new File("path/to/file.abc"));`. – Pshemo Sep 08 '20 at 12:45
3

Read & write streams as raw byte[]. Avoid the text streams and readers.

In your last question your were apparently breaking up the files according to 'line'. To replicate that behavior, simply used a fixed size of byte[] to read. Note carefully the warnings in comments to your last question, to check how many bytes are read. A byte[2^16] will not necessarily be filled in a single read.

Andrew Thompson
  • 168,117
  • 40
  • 217
  • 433
  • Dear Andrew, I am almost tired with this. If you seek into my questions, you may find another question I have asked about something similar to this few months back. So, could you please be kind enough to give me an example code? That will be greatly appreciated :) – PeakGen Jun 02 '12 at 17:24