JAVA not in gzip format error

Question

I have an inputstream and I tried to process it but it gave me this error "not in gzip format" but the file is in gzip format "Content-Encoding: gzip"

protected String readResponse(InputStream is) throws IOException {
StringBuffer string;
int b;
byte[] buffer;
String eol, s = null;
GZIPInputStream gis;
int read;
int index;


eol = new String(new byte[] {(byte)0, (byte)0, (byte)-1, (byte)-1});
buffer = new byte[1];
string = new StringBuffer();
while ( (b = is.read()) > 0 ) {
  buffer[0] = (byte)b;
  s = new String(buffer);
  string.append(s);
  index = string.indexOf(eol);
  if ( index > 0 && index == string.length() - 4 ) {
    break;
  }

}

System.out.println(string);

gis = new GZIPInputStream(is); << here I got the error
buffer = new byte[1024]; 

while ( (read = gis.read(buffer)) > 0 ) {
  string.append(new String(buffer, 0, read));
}
return string.toString();

}

any thoughts? thanks

Answer 1

Seeing this line:

eol = new String(new byte[] {(byte)0, (byte)0, (byte)-1, (byte)-1});

is enough to arrive to a conclusion: you are doomed from the start.

DO NOT USE STRING FOR BINARY DATA.

bytes and chars have no relationship to one another; what you are doing here is roughly equivalent to the following:

final CharsetDecoder decoder = Charset.defaultCharset()
    .newDecoder().onMalformedInput(CodingErrorAction.REPLACE);
final ByteBuffer buf = ByteBuffer.wrap(new byte[]{...});
final CharBuffer cbuf = decoder.decode(buf);
final String eol = new String(cbuf.array());

Note the REPLACE action. Any unmappable byte sequence will trigger the decoder to output the Unicode replacement character, U+FFFD (looks familiar, right?).

Now try and put REPORT instead.

What is more, you use the default charset... Which differs from platform to platform.

Your code should really just read the input stream and return a byte array. use a ByteArrayOutputStream.

And if you want to write to a file directly, it's easy: use Files.copy().

Anyway, fixed that for you:

// Note: return code is byte[]
protected byte[] readResponse(final InputStream in)
    throws IOException
{
    try (
        final InputStream gzin = new GzipInputSream(in);
        final ByteArrayOutputStream out = new ByteArrayOutputStream();
    ) {
        final byte[] buf = new byte[4096];
        int bytesRead;
        while ((bytesRead = gzin.read(buf)) != -1)
            out.write(buf, 0, bytesRead);

        return out.toByteArray();
    }
}

Answer 2

The problem could be you're advancing the file pointer in the input stream before you pass it to GZIPInputStream. GZIPInputStream expects the first few bytes to be a standard header.

Try moving new GZIPInputStream(is); before your while loop

Answer 3

There is so many things wrong in your code..... But lets try anyway. So you have ascii header and after that there shoulbe gzipped part? Gzip file always starts with id bytes. These have the fixed values 'ID1 = 31 (0x1f, \037), ID2 = 139 (0x8b, \213)'. Can you find those from your inputstream. There you should start the gzipstream.

Answer 4

I have tested this with a file composed from a few header lines, followed by an empty line, and an appended gzipped text file. The latter is written, unexpanded, to x.gz and unzipped and read from there, assuming that it is a text file. (If it is a binary file, a BufferedReader is pointless.)

try/with resources and catch should be added, but that's just a technicality.

InputStream is = ...;
StringBuilder lsb = new StringBuilder();
int c = -1;
while( (c = is.read()) != -1 ){
    if( c == '\n' ){
        String line = lsb.toString();
        if( line.matches( "\\s*" ) ){
            break;
        }
        System.out.println( line );
        lsb.delete( 0, lsb.length() );
    } else {
        lsb.append( (char)c );
    }
}
byte[] buffer = new byte[1024];
int nRead = 0;
OutputStream os = new FileOutputStream( "x.gz" );
while ( (nRead = is.read(buffer, 0, buffer.length )) > 0 ) {
    os.write( buffer, 0, nRead );
}
os.close();
is.close();

InputStream gis = new GZIPInputStream( new FileInputStream( "x.gz" ) );
InputStreamReader isr = new InputStreamReader( gis );
BufferedReader br = new BufferedReader(isr);
String line;
while( (line = br.readLine()) != null ){
    System.out.println("line: " + line );
}
br.close();