1

I'm using the zip4j library to zip and unzip files (xml in particular). For an incoming String, I need to write the contents of the String into an xml file, and then zip that file into a zip-file, before transferring the byte[] content of the zip file into an apache camel body object. The zipping part works alright, no issues here.

When I'm trying to read the message back into my application, I get the below error -

net.lingala.zip4j.exception.ZipException: unexpected end of file when reading short buff
    at net.lingala.zip4j.core.HeaderReader.readIntoBuff(HeaderReader.java:1094)
    at net.lingala.zip4j.core.HeaderReader.readCentralDirectory(HeaderReader.java:221)
    at net.lingala.zip4j.core.HeaderReader.readAllHeaders(HeaderReader.java:94)
    at net.lingala.zip4j.core.ZipFile.readZipInfo(ZipFile.java:425)
    at net.lingala.zip4j.core.ZipFile.getFileHeaders(ZipFile.java:688)
    at ZipUtil.unzipFile(ZipUtil.java:230)
    at ZipUtil.unzipFile(ZipUtil.java:218)

What I'm doing essentially is -

  1. Get the byte[] array from the exchange object using the `exchange.getIn().getBody(byte[].class) method
  2. Write the byte[] into a file
  3. Since this is a zip payload, I rename the file to a .zip file
  4. I then try to read the contents of the zip file into a String.

Here's the code -

public String extractMessageFromExchange(Exchange exchange) {
 byte[] bytes = exchange.getIn().getBody(byte[].class);
 File file = null;
 File zipFile = null;

 /*
 * 1. Dump bytes into a file
 * 2. Add ".zip" extension to file
 * Now, Unzip the file (It's convoluted, but that's how we receive payload back from downstream)
 * 3. Use ZipUtil to get path to the unzipped file
 * 4. Use FileUtil to read the contents of the file into a String object
 */

 try {
        String filePath = zipFileGenPath + zipFileGenSuffix + hyphen + 
 incoming;
        file = new File(filePath);
        FileUtils.writeByteArrayToFile(file, bytes);
        zipFile = new File(filePath+zipFileGenExt);
        boolean renamed = file.renameTo(zipFile);
        log.info("Temp zip file created at - " + zipFile.getPath());
                     String underlyingXMLFilePath = 
 unzipFile(zipFile.getPath(), zipFileGenPath); //gives the path
        incomingMessage = FileUtils.readFileToString(new 
 File(underlyingXMLFilePath));
         return incomingMessage;
     } catch (Exception e) {
         log.error("Exception occurred while reading byte[] payload into zip 
 file : ", e.getMessage());
         log.error("Error stacktrace: ", e);
     } finally {
         deleteFile(zipFile);
 }
}

And then the culprit code -

public String unzipFile(final String sourceZipFile, String updateToDir) throws Exception {
        ZipFile zipFile = new ZipFile(sourceZipFile);
        return unzipFile(zipFile, updateToDir);
    }

    public String unzipFile(ZipFile zipFile, String updateToDir) throws Exception {
        ZipInputStream is = null;
        OutputStream os = null;
        ZipParameters parameters = new ZipParameters();
        parameters.setCompressionMethod(Zip4jConstants.COMP_DEFLATE);
        parameters.setCompressionLevel(Zip4jConstants.DEFLATE_LEVEL_NORMAL);
        parameters.setEncryptFiles(false);

        try {
            List fileHeaderList = zipFile.getFileHeaders();
            for (int i = 0; i < fileHeaderList.size(); i++) {
                FileHeader fileHeader = (FileHeader) fileHeaderList.get(i);
                if (fileHeader != null) {
                    String outFilePath = updateToDir + File.separator + fileHeader.getFileName();
                    File outFile = new File(outFilePath);
                    is = zipFile.getInputStream(fileHeader);
                    os = new FileOutputStream(outFile);
                    int readLen = -1;
                    byte[] buff = new byte[4096];
                    while ((readLen = is.read(buff)) != -1) {
                        os.write(buff, 0, readLen);
                    }
                    return outFilePath;
                }
            }
        } finally {
            IOUtils.closeQuietly(is);
            IOUtils.closeQuietly(os);
        }
        return null;
    }

The result ->

ZipUtil - Exception occurred while reading byte[] payload into zip file : 
ZipUtil - Error stacktrace: 
net.lingala.zip4j.exception.ZipException: unexpected end of file when reading short buff
    at net.lingala.zip4j.core.HeaderReader.readIntoBuff(HeaderReader.java:1094)
    at net.lingala.zip4j.core.HeaderReader.readCentralDirectory(HeaderReader.java:221)
    at net.lingala.zip4j.core.HeaderReader.readAllHeaders(HeaderReader.java:94)
    at net.lingala.zip4j.core.ZipFile.readZipInfo(ZipFile.java:425)
    at net.lingala.zip4j.core.ZipFile.getFileHeaders(ZipFile.java:688)
    at ZipUtil.unzipFile(ZipUtil.java:230)
    at ZipUtil.unzipFile(ZipUtil.java:218)
    at ZipUtil.extractMessageFromExchange(ZipUtil.java:96)

Any idea what's happening? PS. I'm trying these in a Unix box.

GopherGopher
  • 376
  • 1
  • 2
  • 16
  • 1
    Looks like you are using v1.x of Zip4j. Please try upgrading to the latest version (2.1.2). You will have to do some minor code changes as the api has changed with v2.x. The upside is that with v2.x you don't have to create a file to read the content back. You can use `ByteArrayOutputStream` and pass in your byte array into that stream, and then pass in the `ByteArrayOutputStream` to `ZipInputStream`. Please look at the stream section in Zip4j's readme to see more elaborate example. Let me know if you still face this issue with 2.x. – Srikanth Reddy Lingala Aug 20 '19 at 07:58
  • And also, please make sure that the byte array you pass in is the same content as the byte array you get back. Try this with a single small file so you are able to print and compare the byte arrays. – Srikanth Reddy Lingala Aug 20 '19 at 08:01
  • Yes, you're right - I'm using `1.3.2` version. Would there be an issue if the downstream system also uses `1.3.2` of `zip4j`? I am presuming it'll be backward compatible, but still.. – GopherGopher Aug 20 '19 at 08:23
  • Maybe before upgrading the first thing you can do is to check the byte array content. Most likely you did not get back the complete byte array content that you persisted. – Srikanth Reddy Lingala Aug 20 '19 at 08:31
  • The files are required for some processing purpose later on, so I presume baos won't add much. Is this a platform issue? I mean, I'm able to read it properly in my local Windows environment, but somehow it fails in a Unix box. It might also be worth noting that the content in local in unix is not the same, though both use `zip4j` to zip and unzip. – GopherGopher Aug 20 '19 at 09:49
  • What this error message means is that a part of the zip file content is missing. Zip4j tries to read through the file, but reached end of file when it expected to read more content. I think that you are losing some part of the file when you read it back, or you are not putting in the complete content of the file. As I mentioned earlier you should read it compare the bytes that are being put in and put out. If they are the same, then you are not putting the complete content on the file. If they are not same, you also know the cause of the issue. – Srikanth Reddy Lingala Aug 20 '19 at 11:30
  • Can a `zip4j` zip file be generated from a zipped byte array? I think that's the pain point here. I am generating a zip file, then converting it into a `byte[]` before sending over a queue. On the other side, when I receive the `byte[]`, simply writing those bytes to a file and renaming the file to `file.zip` (as can be seen in code above) doesn't generate a valid zip4j zip file. What am I missing here? – GopherGopher Aug 21 '19 at 00:05
  • Not sure how to do it in camel but you would want to ensure it is put as a jms bytes message vs a JMS text message. With text things can be converted causing the appearance of corruption. If you don't have that option them you could base 64 encode it. – JoshMc Aug 21 '19 at 05:45
  • @MananShah As I have mentioned several times above, did you compare the byte content (not just size but also byte content) that is being sent in vs the content that you get back? After you get back and put that content in a file, are you able to extract it with other zip tools (with both encrypted and unencrypted byte content)? These are the points you should be looking at. Unfortunately, it will not help me help you if you repeat the same thing over and over again. – Srikanth Reddy Lingala Aug 21 '19 at 05:55

1 Answers1

0

I managed to solve it. The problem wasn't with the compression logic or with zip4j, but a naughty little "convertBodyToString" method just before the decompression kicked in. That essentially led to corruption of the original zipped message, due to which the decompression couldn't succeed.

For all you know, a stray ~ character at the end of the text spoiled it.

GopherGopher
  • 376
  • 1
  • 2
  • 16
  • `String` is not a container for binary data. – user207421 Feb 03 '20 at 05:41
  • Agreed, but there were other operations that were being carried out by legacy code before (which didn't have binary data, just plain old `String`). Needed to weed them out. The fact that there wasn't any exception handling for the said piece of code made it worse. Peace! – GopherGopher Feb 03 '20 at 07:38