streamWriter.Write hangs when writing large string, but API operation completes successfully

Question

I have a xamarin forms app that makes API calls, and when the api calls involve very small json strings they work perfectly. An example payload that works is:

{"command":"","route":"check_password","password":"password","sqlctype":"","service":"",data:""}

An example payload that doesn't work is:

{"command":"","route":"run_script","password":"","sqlctype":"","service":"Image_Processor",data:"base64 string that is 2MB in size"}

However, with the second payload the streamWriter doesn't succeed in writing it and hangs indefinitely. My code is here:

            using (var streamWriter = new StreamWriter(httpWebRequest.GetRequestStream()))
            {
                string json = "{\"command\":\"\"," +
                              "\"route\":\""+route+"\","+
                              "\"password\":\""+password+"\"," +
                              "\"sqlctype\":\"\"," +
                              "\"service\":\""+service+"\"," +
                              "\"data\":\""+data+"\"}";

                streamWriter.Write(json);
            }

But the interesting part is that the API operation completes successfully in both cases

The cloudwatch logs for a run with the first payload are as follows:

START RequestId: ee583521-f17d-4143-9c27-031ab781ddae Version: $LATEST
END RequestId: ee583521-f17d-4143-9c27-031ab781ddae
REPORT RequestId: ee583521-f17d-4143-9c27-031ab781ddae Duration: 7.01 ms Billed Duration: 8 ms Memory Size: 128 MB Max Memory Used: 93 MB XRAY TraceId: 1-611ac895-4ba33ffa6e181b6e66821bfc...

With a returned body of

"T"

The cloudwatch logs for a run with the second payload are as follows:

START RequestId: d78b16de-1a0b-4b35-877a-2fd6437dcd71 Version: $LATEST
END RequestId: d78b16de-1a0b-4b35-877a-2fd6437dcd71
REPORT RequestId: d78b16de-1a0b-4b35-877a-2fd6437dcd71 Duration: 4159.55 ms Billed Duration: 4160 ms Memory Size: 128 MB Max Memory Used: 93 MB XRAY TraceId: 1-611ac940-61cca9b762d1a2ff6...

With a returned body of

"[{\"typ\": \"barcode\", \"outp\": \"0882658000546\", \"ymax\": \"0.6736111111111112\", \"ymin\": \"0.550843253968254\", \"xmax\": \"0.6736111111111112\", \"xmin\": \"0.3488756613756614\"}, {\"typ\": \"barcode\", \"outp\": \"A-0020-Z\", \"ymax\": \"0.4595734126984127\", \"ymin\": \"0.40153769841269843\", \"xmax\": \"0.6008597883597884\", \"xmin\": \"0.3194444444444444\"}, {\"typ\": \"barcode\", \"outp\": \"A-0010-Z\", \"ymax\": \"0.34077380952380953\", \"ymin\": \"0.29092261904761907\", \"xmax\": \"0.5929232804232805\", \"xmin\": \"0.328042328042328\"}]"

Since the API is clearly running and running successfully in both cases, why might the app be hanging at

streamWriter.Write(json)

with only the second payload?

It may also be useful to note that I tried to do this async too and it hung on client.postasync regardless of the payload (neither payload didn't hang). Code:

        async Task<string> asyncAPI(string route, string password = "", string data = "", string service = "")
        {
            HttpClient client = new HttpClient();
            Uri uri = new Uri("https://uidvxrcio9.execute-api.us-east-2.amazonaws.com/default/Blueline_General_API");
            string json = JsonConvert.SerializeObject(new APIrequest(default, route, password, default, service, "dawdasdqwd2"));
            StringContent content = new StringContent(json, Encoding.UTF8, "application/json");
            client.DefaultRequestHeaders.Add("x-api-key", "key");
            HttpResponseMessage response = null;
            response = await client.PostAsync(uri, content);
            return await response.Content.ReadAsStringAsync();
        }

Decoding base64 of this size is a CPU bound operation by nature. Mobile Phones have a very token CPU. So this might actually be how long it takes. Have you tried some intermediate Data sizes that are bigger then 0, but less then 2 MiB? — Christopher, Aug 13 '21 at 18:18
Decoding is not done on the phone. The encoding is done, and then the resulting base64 string is written to the request stream of an httpWebRequest and sent to an API. — Owen Burns, Aug 13 '21 at 18:26
@OwenBurns "hangs indefinitely" ? Is this work being done on non-UI thread? Does the OS kill the app via an ANR? What is the current process status? What is in `logcat`? — SushiHangover, Aug 14 '21 at 00:12
2 MB is not large. The problem almost certainly isn’t the stream writer. As a test, first read the request stream into a string in memory. I bet that’s what hangs. More importantly, do this on a background thread - web response must have taken too long, caused some problem being on main thread. Did you look in debug output for warnings or errors? — ToolmakerSteve, Aug 14 '21 at 19:00
There is nothing but threads opening and then closing in the debug window. I'm not sure how to check the process status and I am not sure what ANR means. It is run in a blocking thread. I added the logcat output to the question. — Owen Burns, Aug 16 '21 at 18:49

score 1 · Answer 1 · answered Aug 13 '21 at 20:13

1

Why are you trying to serialize by your own code? Use the Newtonsoft.Json nuget package and use the methods in that. When you install that they you can easily serialize all the public properties of a class. The code to serialize one of your classes is like:

JsonConvert.SerializeObject(myCompany)

If you are trying to serialize raw data....you can make a byte[] property in your class like:

public byte[] MyBytes {get; set;}

answered Aug 13 '21 at 20:13

ashlar64

1,054
1
9
24

3

while this is a good point, it should have no bearing on whether or not the `StreamWriter.Write()` is failing – Jason Aug 13 '21 at 20:33
Perhaps....but is this section of the text above valid json? ,data:"base64 string that is 2MB in size"} The data field doesn't have quotes.... – ashlar64 Aug 15 '21 at 21:05
I am using the json viewer plugin in Notepad++ and it will not format this text as json: {"command":"","route":"run_script","password":"","sqlctype":"","service":"Image_Processor",data:"base64 string that is 2MB in size"} It says the json is invalid. – ashlar64 Aug 15 '21 at 21:08
I'm not sure how the JSON can be invalid when small requests work as intended. Wouldn't incorrect JSON mean the code wouldn't work at all? – Owen Burns Aug 16 '21 at 18:19
I tried this now and it did not solve my problem. – Owen Burns Aug 16 '21 at 19:22
Does it work with a small amount of data when using Json.net? And it doesn't work with alot of data when using json.net? Also when we are talking alot of data....about how much are we talking? – ashlar64 Aug 18 '21 at 20:49

streamWriter.Write hangs when writing large string, but API operation completes successfully

1 Answers1