I'm trying to retrieve a gzipped XML file from a URL. My problem is, when decompressing it using GZipInputStream in production, the content I get seems to be truncated. I can only read up to a relatively small part of the xml.
URL feedURL = new URL(url);
HttpURLConnection connection = (HttpURLConnection) feedURL.openConnection();
GZIPInputStream input = new GZIPInputStream(connection.getInputStream());
BufferedReader br = new BufferedReader(new InputStreamReader(input));
StringBuffer sb = new StringBuffer();
String line = "";
while((line = br.readLine()) != null) {
sb.append(line);
}
log.info(sb.toString());
This code always reads up to a small portion of the xml only. Running it multiple times displays the exact same result. I already tried downloading the binary content first by using a ByteArrayOutputStream. The size of the array is exactly the same as the file I downloaded. However, when I use GZipInputStream to decompress the byte array, I still get the same truncated string.
By the way, everything works fine in local so I'm led to believe the GZipInputStream is somehow limited in GAE. Does anyone have any idea how I might be able to go around this?
