I have to handle an exception in Polly where I need to Compare the data in the result set with the data in Cosmos Database and then Retry

Question

For every 200 status code that we're receiving from the gremlin query, I want to check if the result vertex matches with the vertex that needed to be updated. And if it doesn't match then do a retry.

public async Task<string> ExecuteUpdateQueryAsync(string query, bool ignoreConflict = true)
{
    try
    {
        var repsonse = await Policy
                          .Handle<ResponseException>(x => (long)x.StatusAttributes["x-ms-status-code"] != (long)HttpStatusCode.Conflict)
                          .OrResult<ResultSet<dynamic>>(r => r.StatusAttributes["x-ms-status-code"] == 200)
                          .WaitAndRetryAsync(3, retryAttempt => TimeSpan.FromSeconds(Math.Pow(2, retryAttempt)))
                          .ExecuteAsync(() => gremlinClient.SubmitAsync<dynamic>(query));

        logger.LogDebug(query + JsonConvert.SerializeObject(repsonse.StatusAttributes));
        if (repsonse.Count > 0)
        {
            return JsonConvert.SerializeObject(repsonse);
        }
        return default;
    }
    ...
}

EDIT:

The response we get from the Gremlin client looks something like : (hope this helps)

g.V(['1111|2222|3333','1111|2222|3333']).Property('Transaction_Quantity',4500).Property('Transaction_Type','Repack'){"x-ms-status-code":200,"x-ms-activity-id":"a4af8faf-4aa9-4ae2-8dd8-797bdbd80a97","x-ms-request-charge":34.64,"x-ms-total-request-charge":34.64,"x-ms-server-time-ms":7.6626,"x-ms-total-server-time-ms":7.6626}

I want to be able to compare the transaction quantity in this response with the one that we are trying to update.

Answer 1

Disclaimer: I'm not familiar with CosmosDb's Gremlin API, so my suggestions regarding the API usage might be incorrect. I try to focus on the Polly side in my post.

---

So, according to my understanding you want to decide whether or not you need to perform further retry attempts based on the response. If one of the fields of the response matches with some value then you shouldn't otherwise you should.

In case of Polly the Handle, HandleResult, Or and OrResult policy builder functions are anticipating synchronous predicates. In other words they are designed to perform simple assertions (like: status code check, existence check, inner exception type check, etc...) Because of this, most of the time they are defined with simple lambda expressions.

If you need to perform more complex / asynchronous logic then you should put that logic next to your to-be-decorated method call. In other words that logic should belong to the to-be-retried function.

---

As I said inside the disclaimer I'm not familiar with the Gremlin API. According to this SO thread you can deserialize the response into a DTO by first serializing the ResultSet<dynamic> then deserializing that into YourDto.

private async Task<ResultSet<dynamic>> IssueGremlinQueryAsync(string query)
{
    ResultSet<dynamic> results = gremlinClient.SubmitAsync<dynamic>(query);
    var parsedResult = JsonConvert.DeserializeObject<YourDto>(JsonConvert.SerializeObject(results));
    //TODO: perform assertion against the parsedResult
    return results;
}

Here you have several options how you propagate back the need for further retries to the policy:

• Use a ValueTuple where one of the items is a flag
• Use a custom exception
• etc.

ValueTuple

private async Task<(bool, ResultSet<dynamic>)> IssueGremlinQueryAsync(string query)
{
    ResultSet<dynamic> results = gremlinClient.SubmitAsync<dynamic>(query);
    var parsedResult = JsonConvert.DeserializeObject<YourDto>(JsonConvert.SerializeObject(results));

    //Replace this with your own assertion 
    bool shouldRetry = parsedResult.TransactionQuantity % 2 == 0; 
    return (shouldRetry, results);
}

The first item in the tuple is the flag, whereas the second is the unparsed results.

With this method you can refactor the related part of your ExecuteUpdateQueryAsync method like this:

public async Task<string> ExecuteUpdateQueryAsync(string query, bool ignoreConflict = true)
{
    //...

    var retryPolicy = Policy
                        .Handle<ResponseException>(x => (long)x.StatusAttributes["x-ms-status-code"] != (long)HttpStatusCode.Conflict)
                        .OrResult<(bool ShouldRetry, ResultSet<dynamic> Response)>(
                            x => x.ShouldRetry || (long)x.Response.StatusAttributes["x-ms-status-code"] == 200)
                        .WaitAndRetryAsync(3, retryAttempt => TimeSpan.FromSeconds(Math.Pow(2, retryAttempt)));

    var response = await retryPolicy.ExecuteAsync(async () => await IssueGremlinQueryAsync(query));
    
    //...
}

Exception

private async Task<ResultSet<dynamic>> IssueGremlinQueryAsync(string query)
{
    ResultSet<dynamic> results = gremlinClient.SubmitAsync<dynamic>(query);
    var parsedResult = JsonConvert.DeserializeObject<YourDto>(JsonConvert.SerializeObject(results));

    //Replace this with your own assertion 
    bool shouldRetry = parsedResult.TransactionQuantity % 2 == 0;
    return !shouldRetry ? results : throw new OperationFailedRetryNeededException("...");
}

If we should not retry then we return with the unparsed results otherwise we throw a custom exception.

With this method you can refactor the related part of your ExecuteUpdateQueryAsync method like this:

public async Task<string> ExecuteUpdateQueryAsync(string query, bool ignoreConflict = true)
{    
    //...

    var retryPolicy = Policy
                        .Handle<ResponseException>(x => (long)x.StatusAttributes["x-ms-status-code"] != (long)HttpStatusCode.Conflict)
                        .Or<OperationFailedRetryNeededException>()
                        .OrResult<ResultSet<dynamic>>( x => (long)x.StatusAttributes["x-ms-status-code"] == 200)
                        .WaitAndRetryAsync(3, retryAttempt => TimeSpan.FromSeconds(Math.Pow(2, retryAttempt)));

    var response = await retryPolicy.ExecuteAsync(async () => await IssueGremlinQueryAsync(query));
    
    //...
}