Android get request returns incomplete data

Question

As the title states, Im using httpget in android, and its working. However, it's returning incomplete data. the call is returning roughly 99kb of json data. However, My function is only producing 38kb and the json string is incomplete.

I'm not getting any errors at all, so it's hard to see why its happening.

size measurements are just me making the call in the browser and saving the output to a file, and doing the same with the output from the function.

here's the code that gets the url:

public String getpage2(){
    HttpClient client = new DefaultHttpClient();
    HttpGet request = new HttpGet(this.request);
    HttpResponse response = null;
    try {
        response = client.execute(request);
    } catch (ClientProtocolException e2) {
        // TODO Auto-generated catch block
        Log.e("getpage",e2.toString());
        e2.printStackTrace();
    } catch (IOException e2) {
        // TODO Auto-generated catch block
        Log.e("getpage",e2.toString());
        e2.printStackTrace();
    }

    String html = "";
    try {
        BufferedReader reader = new BufferedReader(new InputStreamReader(response.getEntity().getContent(), "UTF-8"));
        StringBuilder builder = new StringBuilder();
        for (String line = null; (line = reader.readLine()) != null ; ) {
            builder.append(line).append("\n");
        }
        html = builder.toString();
        Log.d("nr",builder.capacity() + "/" + builder.length());
    } catch (IOException e) {
        // TODO Auto-generated catch block
        Log.e("getpage",e.toString());
        e.printStackTrace();
    }finally{

    }
    return html;
}

any help would be appreciated. I'm at a loss...

Try to implement solution from the similar question: http://stackoverflow.com/questions/9720381/http-get-request-is-missing-data — Roman, Mar 28 '13 at 22:35
Thanks that seems to be on the right track. However, I'm having trouble implementing that, as its reading in byte data and I need a string. how do I convert that? — Chris, Mar 28 '13 at 23:07

score 1 · Accepted Answer · answered Mar 29 '13 at 02:10

1

you may try this to copy the data verbatim and don't worry about missing EOLs:

StringBuilder sb = new StringBuilder();
int len;
char[] buf = new char[20];
while((len = reader.read(buf)) != -1) {
    sb.append(buf, 0, len);
}

answered Mar 29 '13 at 02:10

lenik

23,228
4
34
43

that's better. but still missing some it looks like. First off, I think the logcat is just not able to display all of it because it seems like I'm getting more than its showing. Using your code, I incremented a counter to see how many actual characters get read. it came out to 100,885... now the output from the browser outputs 103,276 characters. Much closer but still not complete. is there a reason why this would be that you can think of? the whole operation is relatively fast, so I'm pretty sure its not timing out. – Chris Mar 29 '13 at 04:03
You're using `BufferedReader`, the speen does not matter, it buffers the data anyway. Could you please save the original data to the file and the received data to another file and post the diff, that might help a lot? – lenik Mar 29 '13 at 04:06
Sure.. that was gonna be my next step anyway since I've reached the limit on wht info I can get from logcat. Ill post what I find when I get a chance to try it. – Chris Mar 29 '13 at 05:38
ok, so this is really a separate issue, but I cant seem to access my written file. the abs path says its in data/data.. but on ddms my data directory wont expand. and on fileexplorer on the device the data directory says its empty. how do I access the file? – Chris Mar 29 '13 at 06:44
use `Environment.getExternalStorage()` to write file to SD card or (if you like) you may root your phone to get access to `/data/data/` =) – lenik Mar 29 '13 at 07:12
nevermind. I'm running it in the emulator to test this part.. I'll have the file whenver the emulator decides to start lol. – Chris Mar 29 '13 at 07:27
lol.. didn't notice you had replied. just out of curiousity, what would be the benefit of rooting my tablet? everybody seems to have a rooted device now. i remember when that was something that was SUPER hard to do – Chris Mar 29 '13 at 07:28
Ok, this worked! it's getting the whole file :-) I guess it was just that logcat was truncating it that was making me leery. I'm getting a different issue now though :( ...getting a json exception when trying to access the object. but the message is too long with all the data, to display in logcat LOL.. Development is such a joy! – Chris Mar 29 '13 at 07:42
@Chris rooting your phone is pointless nowadays, unless you're using it very heavy for development. It was necessary in the early days of Android, but today most of the things you'd like to put in are already there, and rooting is just a large security breach waiting to be exploited by numerous malware. – lenik Mar 29 '13 at 11:56

Android get request returns incomplete data

1 Answers1