HttpURLConnection response is incorrect

Question

When using this code below to make a get request:

private String get(String inurl, Map headers, boolean followredirects) throws MalformedURLException, IOException {

        URL url = new URL(inurl);
        HttpURLConnection connection = (HttpURLConnection) url.openConnection();
        connection.setInstanceFollowRedirects(followredirects); 

        // Add headers to request.
        Iterator entries = headers.entrySet().iterator();
        while (entries.hasNext()) {
            Entry thisEntry = (Entry) entries.next();
            Object key = thisEntry.getKey();
            Object value = thisEntry.getValue();
            connection.addRequestProperty((String)key, (String)value);
        }

        // Attempt to parse
        InputStream stream = connection.getInputStream();
        InputStreamReader isReader = new InputStreamReader(stream ); 
        BufferedReader br = new BufferedReader(isReader );
        System.out.println(br.readLine());

        // Disconnect
        connection.disconnect();

        return connection.getHeaderField("Location");
}

The resulting response is completely nonsensical (e.g ��:ks�6��﯐9�rђ� e��u�n�qש�v��"uI*�W��s)

However I can see in Wireshark that the response is HTML/XML and nothing like the string above. I've tried a myriad of different methods for parsing the InputStream but I get the same result each time.

Please note: this only happens when it's HTML/XML, plain HTML works.

Why is the response coming back in this format?

Thanks in advance!

=== SOLVED ===

Gah, got it!

The server is compressing the response when it contains XML, so I needed to use GZIPInputStream instead of InputSream.

GZIPInputStream stream = new GZIPInputStream(connection.getInputStream());

Thanks anyway!

Which encoding are you using? Note: **Never** use an `InputStreamReader` without specifying an encoding. — Kayaman, Apr 11 '14 at 15:09
@Kayaman I've tried specifying UTF-8, but that made no difference whatsoever. — boiiocks, Apr 11 '14 at 15:12
Then you've looked at the wrong wireshark frames. GZip encoded traffic should look like your "nonsense" data. — hgoebl, Apr 11 '14 at 15:47

score 2 · Answer 1 · answered Apr 11 '14 at 15:11

2

use an UTF-8 encoding in input stream like below

InputStreamReader isReader = new InputStreamReader(stream, "UTF-8");

answered Apr 11 '14 at 15:11

lakshman

2,641
6
37
63

Besides, that doesn't look at all like UTF-8. It could be UTF-16. – Kayaman Apr 11 '14 at 15:15

HttpURLConnection response is incorrect

1 Answers1