Trouble generating over 200MB file into ZIP using ByteArrayOutputStream and ZipOutputStream

Question

I need your help again. My application trying to export user's files (settings, processes, standard files, etc.) as ZIP. This exported ZIP will be used later to import user's files in other system. So far I have done following code:

def files = File.findAllByUser(user)
ByteArrayOutputStream outputStream = new ByteArrayOutputStream()
ZipOutputStream zipOutputStream = new ZipOutputStream(outputStream)

files.each { File it ->
   if (it.fileContent.content) {
      ZipEntry entry = new ZipEntry(user.name + "-" + it.id)
      entry.setSize(it.fileContent.content.length())

      zipOutputStream.putNextEntry(entry)
      zipOutputStream.write(it.fileContent.content.getBytes(1, it.fileContent.content.length() as int))
      zipOutputStream.closeEntry()
   } else {
      log.error("File ${it.id} has no content")
   }
}

I saved the files as xml and then as ZIP in user's backup directory, as the following code do:

ZipEntry entry = new ZipEntry("${user}.xml")
entry.setSize(xml.bytes.length)
zipOutputStream.putNextEntry(entry)
zipOutputStream.write(xml.bytes)
zipOutputStream.closeEntry()
zipOutputStream.close()

Directory backupDir = Directory.findByUserAndIsBackupDirectory(userName, true)

File savedFile = new File(user: user, name: "${userName}.zip", contentType: "application/zip", directory: backupDir).save(flush: true, failOnError: true)
FileContent savedFileContent = new FileContent(file: savedFile).save(flush: true, failOnError: true)
savedFileContent.setDataBytes(outputStream.toByteArray())

So far, it's doing what we want, except that they don't save a file that over 200MB. Do I missed something important here? How can I change my code to also save files that bigger then 200MB?

I've localized the error. It's on this line: savedFileContent.setDataBytes(outputStream.toByteArray()) I'll take a look of that, and give an update later. — blown.eye, Jul 06 '18 at 09:13

score 1 · Answer 1 · answered Jul 06 '18 at 13:53

Chunk the output stream so you're not trying to load everything all at once:

byte[] buffer = new byte[16 * 1024]
File zipFile = new File("myfile.zip")
zipFile.withOutputStream { OutputStream out ->
  ZipOutputStream zipOutputStream = new ZipOutputStream(out)
  File[] fileList = <<the list of files to zip>>
  fileList.each { File file ->
    zipOutputStream.putNextEntry(new ZipEntry(file.getName()))
    def inputStream = new FileInputStream(file)
    int len
    while ((len = inputStream.read(buffer)) > 0) {
      zipOutputStream.write(buffer, 0, len)
    }
    inputStream.close()
    zipOutputStream.closeEntry()
  }
  zipOutputStream.finish()
}

Trouble generating over 200MB file into ZIP using ByteArrayOutputStream and ZipOutputStream

1 Answers1