Deflate and Inflate Java String in Memory Zip Exception Error

Question

I am writing code to deflate and inflate a string in base 64 encode but I am getting the following error:

Exception in thread "main" java.util.zip.ZipException: incorrect header check
    at java.util.zip.InflaterOutputStream.write(InflaterOutputStream.java:284)
    at java.io.FilterOutputStream.write(FilterOutputStream.java:108)
    at serializer.test.SerializerTest.main(SerializerTest.java:43)

My code is:

XsltObject Xslt = new XsltObject();
            Xslt.setXslt(readFile("C:\\codebase\\OverallSystem\\EBE_TEMPERED_XMLS\\bank_timestamp-0.xml"));
            System.out.println("Original String Length: "+ Xslt.getXslt().length());
           //JSONObject jsonObj = new JSONObject( Xslt );
           // System.out.println( jsonObj );
            //System.out.println( "Json Length:" + jsonObj);

            DeflaterOutputStream outputStream;

            for ( int i = 1; i <= 9; ++i ) {
                ByteArrayOutputStream arrayOutputStream = new ByteArrayOutputStream();
                outputStream = new DeflaterOutputStream(arrayOutputStream, new Deflater( i, true ));
                outputStream.write(Xslt.getXslt().getBytes());
                outputStream.close();
                //System.out.println("Deflate (lvl=" + i + ");" + arrayOutputStream.toString("ISO-8859-1"));
                System.out.println("Deflate (lvl=" + i + ");" + arrayOutputStream.toString("ISO-8859-1").length()); 

                String temp = DatatypeConverter.printBase64Binary(arrayOutputStream.toString("UTF-8").getBytes());
                System.out.println(temp);
                System.out.println("Base 64 len: " + temp.length());

                byte[] data =DatatypeConverter.parseBase64Binary(temp);
                ByteArrayOutputStream inflateArrayOutputStream = new ByteArrayOutputStream();
                InflaterOutputStream iis = new InflaterOutputStream(inflateArrayOutputStream, new Inflater());
                iis.write(data);
                iis.close();
                System.out.println("Inflate (lvl=" + i + ");" + inflateArrayOutputStream.toString("ISO-8859-1"));
                System.out.println("Inflate (lvl=" + i + ");" + inflateArrayOutputStream.toString("ISO-8859-1").length());

What am I doing wrong?

score 0 · Answer 1 · answered Mar 30 '16 at 15:41

This fixed all my issues,and is all JDK usage:

package serializer.test;

import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.util.Arrays;
import java.util.zip.*;

import javax.xml.bind.DatatypeConverter;

public class DeflationApp
{
    private String compressBase64(String stringToCompress, int level)
            throws UnsupportedEncodingException
    {
        byte[] compressedData = new byte[1024];
        byte[] stringAsBytes = stringToCompress.getBytes("UTF-8");

        Deflater compressor = new Deflater(level, false);
        compressor.setInput(stringAsBytes);
        compressor.finish();
        int compressedDataLength = compressor.deflate(compressedData);

        byte[] bytes = Arrays.copyOf(compressedData, compressedDataLength);
        return DatatypeConverter.printBase64Binary(bytes);
    }

    private String decompressToStringBase64(String base64String)
            throws UnsupportedEncodingException, DataFormatException
    {
        byte[] compressedData = DatatypeConverter
                .parseBase64Binary(base64String);

        Inflater deCompressor = new Inflater();
        deCompressor.setInput(compressedData, 0, compressedData.length);
        byte[] output = new byte[100000];
        int decompressedDataLength = deCompressor.inflate(output);
        deCompressor.end();

        return new String(output, 0, decompressedDataLength, "UTF-8");
    }

    public static void main(String[] args) throws DataFormatException,
            IOException
    {
        DeflationApp m = new DeflationApp();
        String strToBeCompressed = readFile(
                "C:\\codebase\\OverallSystem\\MappingMapToEBECommon.xslt")
                .trim();
        for (int i = 1; i <= 9; ++i)
        {
            String compressedData = m.compressBase64(strToBeCompressed, i);
            String deCompressedString = m.decompressToStringBase64(compressedData);

            System.out.println("Base 64:");
            System.out.println("Original Length with level("+i+"): " + strToBeCompressed.length());
            System.out.println("Compressed with level("+i+"): " + compressedData.toString());
            System.out.println("Compressed with level("+i+") Length: " + compressedData.toString().length());
            System.out.println("Decompressed with level("+i+"): " +
                    + deCompressedString.length());
            System.out.println("Decompressed with level("+i+"): " + deCompressedString);
        }

        for (int i = 1; i <= 9; ++i)
        {
            byte[] compressedData  = m.compress(strToBeCompressed, i);
            String deCompressedString = m.decompressToString(compressedData);

            System.out.println("Without Base 64:");
            System.out.println("Original Length with level("+i+"): " + strToBeCompressed.length());
            System.out.println("Compressed with level("+i+"): " + new String(compressedData));
            System.out.println("Compressed with level("+i+") Length: " + new String(compressedData).length());
            System.out.println("Decompressed with level("+i+"): " +
                    + deCompressedString.length());
            System.out.println("Decompressed with level("+i+"): " + deCompressedString);
        }

    }

    private byte[] compress(String stringToCompress, int level) throws UnsupportedEncodingException
    {
        byte[] compressedData = new byte[1024];
        byte[] stringAsBytes = stringToCompress.getBytes("UTF-8");

        Deflater compressor = new Deflater(level, false);
        compressor.setInput(stringAsBytes);
        compressor.finish();
        int compressedDataLength = compressor.deflate(compressedData);

        return Arrays.copyOf(compressedData, compressedDataLength);
    }

    private String decompressToString(byte[] compressedData) throws UnsupportedEncodingException, DataFormatException
    {   
        Inflater deCompressor = new Inflater();
        deCompressor.setInput(compressedData, 0, compressedData.length);
        byte[] output = new byte[100000];
        int decompressedDataLength = deCompressor.inflate(output);
        deCompressor.end();

        return new String(output, 0, decompressedDataLength, "UTF-8");
    }

    public static String readFile(String file) throws IOException
    {
        BufferedReader reader = new BufferedReader(new FileReader(file));
        String line = null;
        StringBuilder stringBuilder = new StringBuilder();
        String ls = System.getProperty("line.separator");

        try
        {
            while ((line = reader.readLine()) != null)
            {
                stringBuilder.append(line);
                stringBuilder.append(ls);
            }

            return stringBuilder.toString();
        }
        finally
        {
            reader.close();
        }
    }

}

Mark O'Donohue · Answer 2 · 2017-07-20T06:09:20.770

I too had memory problems with DeflaterOutputStream - it works if you let it use the default constructor. This works fine :

for (Entry<String, String> entry : valueMap.entrySet()) {
    String key = entry.getKey();
    String value = entry.getValue();                       
    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    DeflaterOutputStream dos = new DeflaterOutputStream(baos);
    try { 
        dos.write(value.getBytes());
        dos.flush();
        dos.close();
    }
    catch (IOException e) {
        throw new RuntimeException(e);
    }
    byte[] zipData = baos.toByteArray();
    zipValueMap.put(key, zipData);
}

But change that to :

ByteArrayOutputStream baos = new ByteArrayOutputStream();
Deflater deflater = new Deflater(Deflater.BEST_SPEED); 
DeflaterOutputStream dos = new DeflaterOutputStream(baos, deflater);

And that gives me memory leak in JVM C code that takes up 80g and crashes my mint system. So why would the default constructor work and yet when I pass my own deflator in it fails so badly :

Decoding DeflaterOutputStream (java 1.8_40) I find some special code in the close method :

public void close() throws IOException {
    if (!closed) {
        finish();
        if (usesDefaultDeflater)
            def.end();
        out.close();
        closed = true;
    }
}

I guess they put that in to workaround problem with deflater.

The best solution was to call it explicitly in the loop :

try { 
    dos.write(value.getBytes());
    dos.flush();
    dos.close();
    deflater.end();
}

And no more memory leak. It's a bad memory leak as well, since it is from the C side, so it never threw a JVM error, it just chewed up all the 40g of ram I had, and then started on the swap space. I had to ssh into the box and kill it.

Deflate and Inflate Java String in Memory Zip Exception Error

2 Answers2