6

I'm trying to use HttpURLConnection on Gingerbread+ Android devices and am having trouble with the gzip encoding. According to the documentation

"In Gingerbread, we added transparent response compression. HttpURLConnection will automatically add this header to outgoing requests, and handle the corresponding response:

Accept-Encoding: gzip"

The problem is that this is not actually happening. The Accept-Encoding: gzip header is not getting added at all. If I add it manually, I would then expect the decompressing part of it to work by connection.openInputStream() to automatically return a GZipInputStream but it just returns a plain InputStream.

Has anyone experienced this? I havent seen any posts of this happening so its very odd. The project is compiled against API 17 so that shouldnt be a problem and the device is running 4.3.

Thanks.

Kyle Fowler
  • 151
  • 1
  • 1
  • 5
  • `The Accept-Encoding: gzip header is not getting added at all.` how did you verify this ? – njzk2 Sep 19 '13 at 15:38
  • By the response not returning Content-Encoding: gzip. Its also not in urlConnection.getRequestProperties(). Adding Accept-Encoding: gzip through addRequestProperty causes the response to come back gzipped, so i decided it was not being added. – Kyle Fowler Sep 19 '13 at 20:08

4 Answers4

14

I had the same problem and it was related to HTTPS. If you call:

URL url = new URL("https://www.example.com");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();

you actually get an instance of HttpsURLConnection (a subclass of HttpURLConnection) that does NOT handle gzip automatically. In those cases you can:

conn.setRequestProperty("Accept-Encoding", "gzip");
...
InputStream inStream = new GZIPInputStream(conn.getInputStream());
idrosid
  • 7,983
  • 5
  • 44
  • 41
  • 1
    HttpURLConnection adds gzip header by default. – Nik Jun 24 '15 at 14:08
  • @Nik, apparently you didn't read the answer before you comment. The whole point is that HttpURLConnection adds the header but HttpsURLConnection (with an "s") does NOT. And that's what caused the problem mentioned in the question. The subclass behaves differently. – idrosid Jun 25 '15 at 10:59
  • 1
    It seems that the header was indeed not added automatically in Gingerbread for https, but it is properly added for https since Ice Cream Sandwich. In ICS, http and https implementations share the same HTTP engine. – BladeCoder Feb 20 '17 at 14:06
  • This is probably because encrypting compressed data could leak the data - i.e. a compressed connection might be insecure. See the well-known "CRIME" or "BREACH" attack. – user253751 Jan 03 '20 at 22:00
3

Since Android Gingerbread (http) and ICS (http + https), when you use Http(s)URLConnection, Android adds the Accept-Encoding: gzip header automatically, but only if you didn't add it yourself first.

In that case, if the server returns gzip-encoded content, you will automatically get a GZIPInputStream and the Content-Encoding: gzip header will be removed from the response. That's why it's called transparent: so the client cannot tell the difference between compressed or uncompressed responses by looking at the stream content or the headers.

If you add the Accept-Encoding: gzip header manually yourself however, then transparent gzip handling is disabled for that request and if the server returns gzip-compressed content, then you need to look for the Content-Encoding: gzip header and create the GZIPInputStream yourself to decompress content.

BladeCoder
  • 12,779
  • 3
  • 59
  • 51
2

I tested with a few of my devices and HttpURLConnection is adding Accept-Encoding: gzip to the headers.

Have you tried configuring Fiddler for your Android devices to verify http headers? Perhaps your server does not support compression.

Andy Gaskell
  • 31,495
  • 6
  • 74
  • 83
  • The server does compress, manually adding the header and opening the input stream with new GZipInputStream(conn.openInputStream()) does the correct thing. Removing all manually handling just returns a standard response and the headers are missing accept-encoding. – Kyle Fowler Sep 19 '13 at 17:47
2

I've experienced this myself and also found that it worked if I added the header and used GZIPInputStream manually. However, we were setting our headers manually (had a custom header we had to add).

The Android documentation says "By default, this implementation of HttpURLConnection requests that servers use gzip compression." So I'm guessing that the default only applies if you don't manually set the headers.

Swampie
  • 165
  • 1
  • 1
  • 9