Getting content from website with HttpURLConnection (Android API 18)

Question

How to get content from website with HttpURLConnection for android APIs 18 and above? The code is working fine for API 23, but inputstream is returning odd values for API 18. This is what I get when I try to read data from URL with API 18:

TTP/1.1 200 OK Content-Type: application/json; charset=utf-8 Access-Control-Allow-Origin: * Cache-Control: no-cache, no-store, max-age=0, must-revalidate Pragma: no-cache Expires: Mon, 01 Jan 1990 00:00:00 GMT Date: Wed, 03 Aug 2016 19:01:33 GMTContent-Encoding: gzip P3P: CP="This is not a P3P policy! See https://support.google.com/accounts/answer/151657?hl=en for more info." P3P: CP="This is not a P3P policy! See https://support.google.com/accounts/answer/151657?hl=en for more info." X-Content-Type-Options: nosniff X-Frame-Options: SAMEORIGIN X-XSS-Protection: 1; mode=block Server: GSE Set-Cookie: NID=83=Nb29w9eQo3Fdx_bQuj6YbdLwSfxjuQT4f1Lcb87IbTXQqdGGh6OyuxxB0XGWxNIiAfMdCePDtDb5P9vMYQvbln7svacSJjFkWnU6-B4AN9vLHHY4RUdL3Xny7zSmE8Lm;Domain=.googleusercontent.com;Path=/;Expires=Thu, 02-Feb-2017 19:01:33 GMT;HttpOnly Set-Cookie: NID=83=Z9EmVPVCfKYu4FrAHTVHDPMNM80s23cO6P1VqJAocZHnrQb8QFPKW9BLjQGu5xKOwtqNaT38gTZVJm1zmbT7tVhZAYCQlaSb7dRiSTcqQ71a41cIs4l67RxEkOjXfttC;Domain=.googleusercontent.com;Path=/;Expires=Thu, 02-Feb-2017 19:01:33 GMT;HttpOnly Alternate-Protocol: 443:quic Alt-Svc: quic=":443"; ma=2592000; v="36,35,34,33,32,31,30" Transfer-Encoding: chunked 00000001 00000001 � 00000001 00000001 �� 00000001 �� 00000001 �� 00000001 ��

What would be the reasoning behind this? I can provide the code if needed. I'm fairly new with this and can't find the answer anywhere.

To get response from URL I use

private String downloadUrl(String urlString) throws IOException {
        InputStream is = null;
        try {
            URL url = new URL(urlString);
            HttpURLConnection conn = (HttpURLConnection) url.openConnection();
            conn.setReadTimeout(10000 /* milliseconds */);
            conn.setConnectTimeout(15000 /* milliseconds */);
            conn.setRequestMethod("GET");
            conn.setDoInput(true);
            // Starts the query
            conn.connect();
            int responseCode = conn.getResponseCode();
            is = conn.getInputStream();
            String contentAsString = convertStreamToString(is);
            return contentAsString;
        } finally {
            if (is != null) {
                is.close();
            }
        }
    }

and the method which converts the stream into string

private String convertStreamToString(InputStream is) {
        BufferedReader reader = new BufferedReader(new InputStreamReader(is));
        StringBuilder sb = new StringBuilder();
        String line = null;
        try {
            while ((line = reader.readLine()) != null) {
                sb.append(line + "\n");
            }
        } catch (IOException e) {
            e.printStackTrace();
        } finally {
            try {
                is.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
        return sb.toString();
    }

@Olayinka the request url is https://script.googleusercontent.com/macros/echo?user_content_key=Na8tXX--5d741fBF3qXgxQ-jgTioUWNF_qRUfaMTI2u2l1ul_HUnJg3AraLc-Zg_-7qR4z385MV_8HoU9qi8lV3nVlpVdzNbOJmA1Yb3SEsKFZqtv3DaNYcMrmhZHmUMWojr9NvTBuBLhyHCd5hHa1ZsYSbt7G4nMhEEDL32U4DxjO7V7yvmJPXJTBuCiTGh3rUPjpYM_V0PJJG7TIaKp8cGSZx_xYuyJMXUeN2rZkXdqYzuG_R6Jippy430I3ogteoSsJwqaprbPnRddBTX1zEidwIB2AQJIW1xFZ9DoEE&lib=MbpKbbfePtAVndrs259dhPT7ROjQYJ8yx — andreas, Aug 03 '16 at 19:34
I convert it to JSONObject later on and it all works perfectly for API 23. — andreas, Aug 03 '16 at 19:36
Can we also see the code where you call the url and extract data — Olayinka, Aug 03 '16 at 19:37
@Olayinka https://github.com/AndreasLoL/NutikorvAlphaRelease/blob/master/app/src/main/java/com/nutikorv/andreas/nutikorvalpha/Objects/ProductsFromURL.java this is the class I use to extract data. — andreas, Aug 03 '16 at 19:40
It doesn't work that way here, you kind of have to add sections of your code connected to the problem to your question. — Olayinka, Aug 03 '16 at 19:41
@Olayinka I don't encode the URL, I just pass it to the AsyncTask class and it goes straight to the urlString (The same URL I sent before). — andreas, Aug 03 '16 at 19:58
http://stackoverflow.com/questions/16905782/htc-one-bug-parts-of-http-header-appearing-in-urlconnection-inputstream — Olayinka, Aug 03 '16 at 20:28

score 1 · Accepted Answer · answered Aug 03 '16 at 20:04

I have had a similar problem.. The problem is that you are using an HttpUrlConnection on an https request. So all of your data is going to be encrypted. I would recommend using some other HttpRequest library like http-request by kevinsawicki: https://github.com/kevinsawicki/http-request. This will have easy support for getting headers, bodies (Json, XML, Plaintext) and other meta data about the request.

How to add the maven In android studio open up the project viewer and open build.gradle (app) and add this line in the dependencies list

compile 'com.github.kevinsawicki:http-request:6.0'

Now open up the (project) build.gradle and make sure under repositories you have this

mavenCentral()

Example get request

HttpRequest req = HttpRequest.get("https://google.com");
req.trustAllCerts();
req.trustAllHosts(); //If you are having certificate problems
int code = req.code();
String body = req.body();
Log.d("CODE:", String.valueOf(code));
Log.d("BODY:", body);

Best regards, David

score 0 · Answer 2 · answered Aug 03 '16 at 21:27

0

you might need

 in = new InputStreamReader(
                    httpResponseCode == HttpURLConnection.HTTP_OK ? conn.getInputStream() : conn.getErrorStream(),
                    "UTF-8");

answered Aug 03 '16 at 21:27

Jason

231
3
14

Getting content from website with HttpURLConnection (Android API 18)

2 Answers2