How can I use Polly to retry x number of times based on response content and then return the response?

Question

In my app I am using the Polly library to call an API.

The API can return warnings and errors in the response. For some of these warnings I want to retry 2 times and the next time I would like to return the response to the caller.

Can this be done?

Edit:

@StephenCleary pointed out I should just handle the response and not throw an exception.

To check the response I need to await the Content. The following will not compile, can anyone see how I can do this?

static IAsyncPolicy<HttpResponseMessage> GetRetryPolicy()
{
    return HttpPolicyExtensions
            .HandleTransientHttpError()
            .OrResult(async msg =>
            {
               var content  = await msg.Content.ReadAsStringAsync();
               return content.Contains("errorcode123");
            })
            .WaitAndRetryAsync(2, retryAttempt => TimeSpan.FromSeconds(Math.Pow(2, retryAttempt)));
}

Your main thread is being blocked in _httpClient.PostAsync() waiting for a response. You would need a loop around the code to retry. So you would also need to change the await. — jdweng, May 04 '21 at 14:20
@jdweng Nothing is blocking here, what on earth gave you that idea? — DavidG, May 04 '21 at 14:28
`we throw a custom exception` - by throwing the exception, you're dropping the response. What you should do instead is handle the *response* in Polly using a delegate that determines whether to retry. — Stephen Cleary, May 04 '21 at 14:30
Thanks @StephenCleary, are you able to see me how this is done please? — Vinyl Warmth, May 04 '21 at 14:34
@DavidG : var httpResponse = await _httpClient.PostAsync(requestUri, new StringContent(requestJson, Encoding.UTF8, "application/json")); — jdweng, May 04 '21 at 14:55
@jdweng That is not blocking, it's just a standard `await`. You clearly have no idea what Polly is or how it works. — DavidG, May 04 '21 at 14:58
@DavidG : That is a block!!! It waits until a response completes. — jdweng, May 04 '21 at 15:05
@OP I think Stephen's thought was along this: https://github.com/App-vNext/Polly#step-1b-optionally-specify-return-results-you-want-to-handle — Fildor, May 04 '21 at 15:16
@Fildor yes I've been looking at that. The problem I'm now having is that to check the response content I need to await it. I'm going to update my post now — Vinyl Warmth, May 04 '21 at 15:18

score 4 · Accepted Answer · answered May 04 '21 at 15:27

There's a couple parts to this.

First, you don't want to throw an exception if the result has warnings. At that point maybe you want to retry and maybe you don't; the code there can't tell yet. But throwing an exception means the response is discarded, so throwing at this point is not correct.

Instead, that handler should mark the response with a "has warnings" flag. This is possible using HttpRequestMessage.Properties (HttpRequestMessage.Options in .NET 5). Something like this:

private static readonly string IsInternalServerResponseKey = Guid.NewGuid().ToString("N");

...

var httpResponse = ...
var responseContent = ...
if (InternalServerResponse(responseContent))
{
    httpResponse.RequestMessage.Properties[IsInternalServerResponseKey] = true;
}

This way there's a flag attached to the request/response that can be read by other parts of the code, specifically the Polly retry handler.

The other part of the solution is the retry count. Normally, Polly has delegates that determine whether or not it should retry, and those delegates are self-contained - retry exceptions of this type, or retry responses that look like that. In this case, you want to retry a response that matches a certain shape but only if there have not been too many retries, and if there have been too many retries and the response matches a "retry" shape, then you don't want to throw an exception but return the response instead.

This is unusual, but doable. You'll need to capture the "external considerations" (in this case, the retry count) inside the Polly context. Then your retry delegate can extract the retry count from the context and base its decision on that. Something like this should work:

private static readonly string RetryCountKey = Guid.NewGuid().ToString("N");
static IAsyncPolicy<HttpResponseMessage> GetRetryPolicy()
{
    return HttpPolicyExtensions
        .HandleTransientHttpError()
        .OrResult(response =>
        {
            return IsInternalServerResponse() && RetryCount() <= 2;

            bool IsInternalServerResponse()
            {
                if (!response.RequestMessage.Properties.TryGetValue(IsInternalServerResponseKey, out var objValue) ||
                    objValue is not bool boolValue)
                    return false;
                return boolValue;
            }

            int RetryCount()
            {
                if (!response.RequestMessage.GetPolicyExecutionContext().TryGetValue(RetryCountKey, out var objValue) ||
                    objValue is not int intValue)
                    return 0;
                return intValue;
            }
        })
        .WaitAndRetryAsync(2,
            (retryAttempt, _) => TimeSpan.FromSeconds(Math.Pow(2, retryAttempt)),
            (_, _, retryAttempt, context) => context[RetryCountKey] = retryAttempt);
}

I haven't tested this; it's possible there's an off-by-one error between the 2 passed to WaitAndRetryAsync and the 2 used to compare the retryCount.

How can I use Polly to retry x number of times based on response content and then return the response?

Edit:

1 Answers1