GetResponse() and WebException kill my thread

Question

I'm actually facing an issue with a httpWebResponse that stop my thread.

I call an ApiCall method in a looping thread every 2 seconds. This works most of the time. But sometimes the request.GetResponse() throws a WebException and stops my main thread loop. It freezes the application without crashing it. It can be retried 5 times(MaxRetries) without working properly. I don't understand what's going on.

Here's the code part on an ApiCall Method. I don't really know who it works. So picked up some code here and there. I must miss something..., but what?

public T CallWithJsonResponse<T>(string uri, bool hasEffects, params Tuple<string, string>[] headers)
    {
        if (simulate && hasEffects)
        {
            Debug.WriteLine("(simulated)" + GetCallDetails(uri));
            return default(T);
        }

        Debug.WriteLine(GetCallDetails(uri));
        var request = HttpWebRequest.CreateHttp(uri);
        foreach (var header in headers)
        {
            request.Headers.Add(header.Item1, header.Item2);
        }
        HttpWebResponse response = null;
        request.Timeout = 300000;

        for (int i = 0; i < MaxRetries; i++)
        {

            try
            {
                response = null;
                using (response = (HttpWebResponse)request.GetResponse())
                {
                    if (response.StatusCode == HttpStatusCode.OK)
                    {
                        try
                        {
                            using (var sr = new StreamReader(response.GetResponseStream()))
                            {
                                var content = sr.ReadToEnd();
                                var jsonResponse = JsonConvert.DeserializeObject<ApiCallResponse<T>>(content);
                                sr.Close();
                                response.Close();

                                if (jsonResponse.success)
                                {
                                    //GC.Collect();
                                    GC.WaitForPendingFinalizers();
                                    return jsonResponse.result;
                                }
                                else
                                {

                                    Console.WriteLine(new Exception(jsonResponse.message.ToString()));

                                }
                            }
                        }
                        catch (Exception ex)
                        {

                            Console.WriteLine("Error - Call Details=" + GetCallDetails(uri) + "Exeption: " + ex.ToString());

                        }
                    }
                    else
                    {
                        Console.WriteLine("Error - StatusCode=" + response.StatusCode + " Call Details=" + GetCallDetails(uri));

                    }
                }
            }
            catch (WebException wex)
            {
                if (wex.Response != null)
                {
                    Console.WriteLine("ERROR (web exception, response generated): " + Environment.NewLine + new StreamReader(wex.Response.GetResponseStream()).ReadToEnd());
                }
                else
                {
                    Console.WriteLine("ERROR (web exception, NO RESPONSE): " + wex.Message + wex.StackTrace);
                }
                Console.WriteLine("Error - Call Details=" + GetCallDetails(uri));

            }
            catch (Exception ex)
            {
                Console.WriteLine("Error - Call Details=" + GetCallDetails(uri) + "Exception: " + ex.ToString());
            }
        }
        return default(T);
    }

private static string GetCallDetails(string uri)
    {
        StringBuilder sb = new StringBuilder();
        var u = new Uri(uri);
        sb.Append(u.AbsolutePath);
        if (u.Query.StartsWith("?"))
        {
            var queryParameters = u.Query.Substring(1).Split('&');
            foreach (var p in queryParameters)
            {
                if (!(p.ToLower().StartsWith("api") || p.ToLower().StartsWith("nonce")))
                {
                    var kv = p.Split('=');
                    if (kv.Length == 2)
                    {
                        if (sb.Length != 0)
                        {
                            sb.Append(", ");
                        }

                        sb.Append(kv[0]).Append(" = ").Append(kv[1]);
                    }
                }
            }
        }
        return sb.ToString();
    }

I am unclear what issue you are having, and what your question is. — mjwills, Jul 16 '17 at 13:34
Please update your post to include the source code for the `GetCallDetails` method. — mjwills, Jul 16 '17 at 13:43
Well, I want to get the code keeps running after a WebException occurs. But the WebException seems to stop the current Thread and freeze my application. Other exceptions just write some text in the console and the application keep running. The ApiCall isn't mandatory. If it fail, no matter. The next try will be done at the next loop. But with a WebException the next loop never come... — TiB, Jul 16 '17 at 13:44
If you make a webrequest and you get a response, I have found that you should always consume the response if there is one. This means reading the reponse stream to completion. Sometimes the response is found on the WebException in the case of a non-success code. Failure to do this can cause hangs under load. — spender, Jul 16 '17 at 13:50
You mean that I need to read the response inside the WebException? — TiB, Jul 16 '17 at 13:55
You need this to happen in ***all*** branches where a response has reached you. — spender, Jul 16 '17 at 14:03

score 0 · Answer 1 · answered Jul 16 '17 at 13:48

I would suggest changing:

catch (WebException wex)
{
    if (wex.Response != null)
    {
        Console.WriteLine("ERROR (web exception, response generated): " + Environment.NewLine + new StreamReader(wex.Response.GetResponseStream()).ReadToEnd());
    }
    else
    {
        Console.WriteLine("ERROR (web exception, NO RESPONSE): " + wex.Message + wex.StackTrace);
    }
    Console.WriteLine("Error - Call Details=" + GetCallDetails(uri));

}

to:

catch (WebException wex)
{
    try
    {
        if (wex.Response != null)
        {
            Console.WriteLine("ERROR (web exception, response generated): " + Environment.NewLine +
                              new StreamReader(wex.Response.GetResponseStream()).ReadToEnd());
        }
        else
        {
            Console.WriteLine("ERROR (web exception, NO RESPONSE): " + wex.Message + wex.StackTrace);
        }
        Console.WriteLine("Error - Call Details=" + GetCallDetails(uri));
    }
    catch (Exception bob)
    {
        // Ignore exceptions here
    }
}

Then set a breakpoint on the catch (Exception bob) line - to see why your existing catch block is failing.

score 0 · Answer 2 · answered Jul 17 '17 at 12:10

Well, it seems to work properly, I tried to consume every response like you said Spender. And I also put the try catch inside the WebException catch as you said mjwills but it never catch. So I think that consume every response is the solution to my problem. Thank you ! Now I need to monitor the application during one week to see if it's robust.

score 0 · Answer 3 · answered Jul 19 '17 at 08:40

Nope... It didn't changed anything. I created a little methods to check if the thread is still alive after a WebException occurred. And guess what! The try/catch works properly, I receive the WebException. But just a few moments later, the thread is stopped or crashed. Don't know witch one is correct... So I implemented an auto-restart if the thread is killed... It's very ugly, but it works.

What is going on with this request.GetResponse()? Why it kills the thread even if the Exception is caught ? It's very strange...

GetResponse() and WebException kill my thread

3 Answers3