Null message on hTTPResponse.Content.ReadAsStringAsync().Result after 1st foreach loop

Question

I have issue with null result messages when calling a HttpClient getAsync within a foreach loop.

I need to iterate with a list of objects for values to call an API via HttpClient. After the first loop, the HttpResponseMessage's result() comes up with null message.

I tried CacheControl.NoCache = true. It's not working.

public async Task<List<Common.ProgressResponse>> RunAsync()
{
    List<Response> ListOfResponses = new List<Responses>();

    try
    {
        _client.BaseAddress = new Uri([URL]);
        _client.DefaultRequestHeaders.Accept.Clear();
        _client.DefaultRequestHeaders.Clear();
        _client.DefaultRequestHeaders.CacheControl = new CacheControlHeaderValue() { NoCache = true };
        _client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
        _client.DefaultRequestHeaders.Add([KEY], [VALUE]);

        ListOfResponses = await SomeFunction();

    }
    catch (Exception ex)
    {
        //Exceptions here...
    }
    return ListOfResponses;
}


private async Task<List<ListOfResponses>> SomeFunction()
{
    List<Response> responses = new List<Response>();
    string aPISuffix = string.Format("{0}/{1}", [APISUFFIX1], [APISUFFIX2]);

    foreach (Object obj in ListOfObject)
    {
        _client.DefaultRequestHeaders.Add("Param1", obj.Param1);
        if (!string.IsNullOrEmpty(obj.Param2))
            _client.DefaultRequestHeaders.Add("Param2", obj.Param2);

        Response response = new Response();


/*This is where the issue is .begin.*/
        HttpResponseMessage hTTPResponse = await _client.GetAsync(aPISuffix).ConfigureAwait(false);
        string result = hTTPResponse.Content.ReadAsStringAsync().Result;
/*This is where the issue is .end.*/


        if (hTTPResponse.IsSuccessStatusCode)
        {
            response = [Code here...]
            //Codes here...
        }

        responses.Add(response);
    }

    return responses;
}

On: 'string result = hTTPResponse.Content.ReadAsStringAsync().Result;' I would expect the result message would have values as it loops through ListOfObjects.

Why are you using `await` everywhere, but suddenly [`.Result`](https://blog.stephencleary.com/2012/07/dont-block-on-async-code.html) on that one line? — GSerg, Jul 17 '19 at 23:04
You should check the httpResponse status code BEFORE reading the content. You might see some surprises. I can't see how the httpclient is being initiated, but I'd be careful about resetting the default headers, and instead I'd try to specify them on a per request basis in order to avoid multiple requests conflicting with each others — ESG, Jul 17 '19 at 23:16
@ESG the httpClient is a static global in the class. The class gets instantiated with a using() statement. — the.herbert, Jul 18 '19 at 00:17
@praetorean be careful with static httpclients too. If you register this class as a singleton, then your DNS entries will get stale. — Rogala, Jul 18 '19 at 00:19
Solved it... I moved the .DefaultRequestHeaders values inside the foreach() loop. I suspected that they were pilling up at every loop iteration. I had to clear them and set them up before the GetAsync() — the.herbert, Jul 18 '19 at 03:29

score 3 · Answer 1 · edited Sep 17 '21 at 06:59

Update: Per comment

First off, I generally try to avoid using .Result with tasks. I also don't think adding the cache headers is the issue since you are making a server to server call.

One thing I tend to always do is evaluate the response before I try to read it. I also prefer to use the HttpRequestMessage so I can manage the disposal of it.

Here is a quick example demonstrating how call out:

using System;
using System.Net.Http;
using System.Threading.Tasks;

namespace Testing
{
    class Program
    {
        static void Main(string[] args)
        {
            TestAsync().Wait();
        }

        static async Task TestAsync()
        {
            var urls = new string[]
            {
                "https://stackoverflow.com/questions/57084989/null-message-on-httpresponse-content-readasstringasync-result-after-1st-foreac",
                "https://stackoverflow.com/users/2025711/rogala"
            };

            using (var httpClient = new HttpClient())
            {
                foreach (var url in urls)
                {
                    using (var request = new HttpRequestMessage(HttpMethod.Get, url))
                    {
                        Console.WriteLine($"Request: {url}".PadLeft(5,'*').PadRight(5, '*'));
                        var response = await httpClient.SendAsync(request)
                            .ConfigureAwait(false);

                        if (response.IsSuccessStatusCode)
                        {
                            var body = await response.Content.ReadAsStringAsync()
                            .ConfigureAwait(false);
                            Console.WriteLine($"{body.Length}{Environment.NewLine}");
                        }
                        else
                        {
                            Console.WriteLine($"*Bad request: {response.StatusCode}");
                        }
                        
                    }
                }
            }

            Console.ReadKey();
        }
    }
}

One additional thing to keep in mind is socket exhaustion. If this code is running on a server, then you will run into socket exhaustion once your service gets some load. I would highly recommend using an HttpClientFactory to handle the HttpClients.

With respect to why the content string is empty, it could be because the server didn't return content which could be a server issue. The server may have also throttled you, which resulted in no content. I would recommend looking into Polly to handle certain response codes.

What? This doesn't really answer the question, You just fixed some of the minor bugs. If you're claim that using .Result is the issue explain why. — johnny 5, Jul 17 '19 at 23:42
No, I am showing the pattern that does work. As mentioned in a comment, you should not attempt to read the body before checking the result. Additionally, this pattern is different in that a new HttpRequestMessage is generated and sent vs calling get. Finally, I could be wrong here, but the caching is not needed for server to server calls. — Rogala, Jul 17 '19 at 23:51
That comment should be the first part of your answer. If you read this answer from a users part of view its unclear on how or why this would fix this issue — johnny 5, Jul 17 '19 at 23:56

score 0 · Accepted Answer · answered Jul 18 '19 at 04:10

Solved it... I moved the .DefaultRequestHeaders values inside the foreach() loop. I suspected that they were pilling up at every loop iteration. I had to clear them and set them up before the GetAsync()

Updated code here:

public async Task<List<Common.ProgressResponse>> RunAsync()
{
    List<Response> ListOfResponses = new List<Responses>();

    try
    {
        _client.BaseAddress = new Uri([URL]);
        ListOfResponses = await SomeFunction();

    }
    catch (Exception ex)
    {
        //Exceptions here...
    }
    return ListOfResponses;
}


private async Task<List<ListOfResponses>> SomeFunction()
{
    List<Response> responses = new List<Response>();
    string aPISuffix = string.Format("{0}/{1}", [APISUFFIX1], [APISUFFIX2]);

    foreach (Object obj in ListOfObject)
    {
        /*Moved Code .begin.*/
        _client.DefaultRequestHeaders.Accept.Clear();
        _client.DefaultRequestHeaders.Clear();
        _client.DefaultRequestHeaders.CacheControl = new CacheControlHeaderValue() { NoCache = true };
        _client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
        _client.DefaultRequestHeaders.Add([KEY], [VALUE]);      

        /*Moved Code .end.*/

        _client.DefaultRequestHeaders.Add("Param1", obj.Param1);
        if (!string.IsNullOrEmpty(obj.Param2))
            _client.DefaultRequestHeaders.Add("Param2", obj.Param2);

        Response response = new Response();
        HttpResponseMessage hTTPResponse = await _client.GetAsync(aPISuffix).ConfigureAwait(false);

        if (hTTPResponse.IsSuccessStatusCode)
        {
            string result = hTTPResponse.Content.ReadAsStringAsync().Result;        
            response = [Code here...]
            //Codes here...
        }

        responses.Add(response);
    }

    return responses;
}

This is exactly why I like to use the HttpRequestMessage. Have to clear headers seems a bit odd. IMHO - the code is clearer by adding it to the message. It’s also easier to unit test. — Rogala, Jul 23 '19 at 22:53

Null message on hTTPResponse.Content.ReadAsStringAsync().Result after 1st foreach loop

2 Answers2