Is there a way to detect if an incoming serialized object stream is GZIPOutputStream compressed or a simple ObjectOutputStream?

Question

I have a legacy system where servers get slowly updated over a period of weeks. The hierarchy is such:

1
2
3 4 5

1 is the client pc
2 is a master server
3 4 and 5 are servers across the country.

Currently all of these are sending POJO (plain old java objects) back and forth in an uncompressed format. Think OjbectOutputStream() etc.

I'd like to compress the data being serialized over the wire but do it in such a way that only data being received from a query is compressed. The data being sent down is trivial (query filter data).

Only client #1 and master server #2 are updated right away. Servers #3, #4 and #5 could be updated weeks or months apart from each other. I need a way for the server #2 to be able to detect whether the streams coming back from #3, #4 or #5 are compressed and deal with it accordingly (as they get upgraded).

-EDIT- The solution must be unobtrusive for the servers #3, #4, and #5. These servers do not have the concept of resending the data if an exception occurs.

Here is an example of code used by #2 to communicate with #3, #4, or #5:

    // Set the content type to be application/x-java-serialized-object
    connection.setRequestProperty("Content-Type", "application/x-java-serialized-object");

    setupHeaderAttributes(getHttpHeaders());

    setupSessionCookies(getHttpHeaders());

    // Load/add httpHeaders
    addHeadersToConnection(connection, getHttpHeaders());

    // Write the serialized object as post data
    objectoutputstream = new ObjectOutputStream(connection.getOutputStream());
    objectoutputstream.writeObject(obj);
    objectoutputstream.flush();

    // Get ready to receive the reply.
    inputstream = connection.getInputStream();
    setHttpStatus(connection.getResponseCode());

Is this possible? Thank you for your time.

-Dennis

score 1 · Answer 1 · edited May 23 '17 at 12:28

You can read in the stream's header. GZIPOutputStream writes the GZIP header into it the stream before anything else, and in lexical hex, it looks like:

0x1f8b 0800 0000 0000 0000

Source

Note that if your legacy servers aren't using Java's GZIPOutputStream, the last 8 bytes may be different. However, the first 2 bytes will always be 0x1f8b. The remaining header values are just information about where it came from and some other flags used by the GZIP format.

kdgregory · Accepted Answer · 2013-06-06T15:29:49.180

@Puce has half the answer. The other half is to use mark() and reset() to reset the stream if it's not a GZipped stream:

    InputStream in = // stream from server
    in = new BufferedInputStream(in);
    in.mark(1024);

    try {
        in = new GZIPInputStream(in);
    }
    catch (ZipException ex) {
        in.reset();
    }

    // "in" is now ready for use

The BufferedInputStream serves two purposes here: first, I know that it supports mark/reset. Second, it will improve IO performance if the underlying stream is not buffered (although, if it's a socket stream, it will be).

The mark value of 1024 is an arbitrary value. The GZipInputStream constructor should be able to determine if the underlying stream is GZipped by reading the first two characters. However, a GZIP header is 10 bytes long, so it might try to read more. It shouldn't read more than 1024 bytes (and if it does, increase the mark value).

Edit: Since I see from your edit that you're using the Content-Type header, you could also use it to optionally unzip the stream: the new servers would return something like x-application/java-gzipped-serialized-object while the old servers continue to return x-application/serialized-java-object (or whatever it was).

Content types beginning with "x-" are unrestricted; you can use whatever you want, as long as both ends agree.

ah! The idea of the content-type sounds like a great idea. Time to experiment. I'll post results later. Thank you. — D-Klotz, Jun 06 '13 at 15:32
So testing has proven the idea of checking the content-type will allow me to introduce this from the top down. It works. Thanks again. — D-Klotz, Jun 08 '13 at 13:50

score 0 · Answer 3 · answered Jun 06 '13 at 14:45

0

The constructors of GZIPInputStream throw a ZipException if they cannot handle the input stream.

ZipException - if a GZIP format error has occurred or the compression method used is unsupported

answered Jun 06 '13 at 14:45

Puce

37,247
13
80
152

While this answer does, do exactly what I asked, I don't know if it will be practical. The stream that came in, and caused the exception would then cause an EOF type of exception back on the legacy server - would it not? – D-Klotz Jun 06 '13 at 14:52

Is there a way to detect if an incoming serialized object stream is GZIPOutputStream compressed or a simple ObjectOutputStream?

3 Answers3