Exception: Microsoft.Azure.Documents.RequestRateTooLargeException while querying from DocumentDB

Question

y query is something like

this.ProcessRequestSync(() => this.Client.CreateDocumentQuery<Model>(this.DocumentDBCollectionLink).Where(d => d.name.Equals(name) && d.code.Equals(code) && d.Type.Equals(this.documentType) && d.CreatedBy.Equals(myName).ToList<Model>());

 public dynamic ProcessRequestSync(Func<dynamic> getRequest)
{
    var delay = TimeSpan.Zero;
    var minDelayTime = new TimeSpan(0, 0, 1);
    for (;;)
    {
        try
        {
            Thread.Sleep(delay);
            return getRequest();
        }
        catch (DocumentClientException documentClientException)
        {
            var statusCode = (int)documentClientException.StatusCode;
            if (statusCode == 429 || statusCode == 503)
            {
                string errorMessage = string.Format("failed at DocumentDB with {0} status and {1} retry time", statusCode, documentClientException.RetryAfter);
                this.Logger.Log(errorMessage );

                // Back off if the request rate is too large or the service is temporarily unavailable
                delay = TimeSpan.Compare(documentClientException.RetryAfter, minDelayTime) >= 0 ? documentClientException.RetryAfter: minDelayTime;
            }
            else
            {
                throw;
            }
        }
    }
}

This is the method for retry logic when requestRateTooLarge exception raise.

I am not sure, whether it is working fine or not,

I am getting Exception: Microsoft.Azure.Documents.RequestRateTooLargeException while querying and inserting around 4000 records at a time,

I used the same retry logic for inserting, its working fine. I am not getting any error and also successfully inserted all records but unable to get query data.

score 5 · Answer 1 · answered Mar 26 '15 at 15:19

5

You also need a catch block for AggregateException, and check if the AggregateException.InnerException is a DocumentClientException and performs the same check for StatusCode == 429. Since the query execution is asynchronous, you might be getting the throttle exception wrapped inside an AggregateException.

If you could post a full repro, we might able to definitively identify the problem.

answered Mar 26 '15 at 15:19

Aravind Krishna R.

7,885
27
37

3

also worth noting here that when handling throttling exceptions don't wait for Timespan.Zero. Rather look at the response you get back which will have the retry-after value on it. Wait for that timeperiod. – Ryan CrawCour Mar 27 '15 at 02:58

BMac · Accepted Answer · 2016-03-15T15:57:08.737

Based on @aravind Ramachandra and @Ryan CrawCour answers above/below, this is what I am using to get round the issue.

    public async Task SaveToDocDb(dynamic jsonDocToSave)
    {

        using (var client = new DocumentClient(endpoint, authKey))
        {
            var queryDone = false;
            while (!queryDone)
            {
                try
                {
                    await client.CreateDocumentAsync(docCollectionlink, jsonDocToSave);
                    queryDone = true; 
                }
                catch (DocumentClientException documentClientException)
                {
                    var statusCode = (int)documentClientException.StatusCode;
                    if (statusCode == 429 || statusCode == 503)   
                        Thread.Sleep(documentClientException.RetryAfter);
                    else
                        throw;
                }
                catch (AggregateException aggregateException)
                {
                    if(aggregateException.InnerException.GetType() == typeof(DocumentClientException)){

                        var docExcep = aggregateException.InnerException as DocumentClientException;
                        var statusCode = (int)docExcep.StatusCode;
                        if (statusCode == 429 || statusCode == 503)
                            Thread.Sleep(docExcep.RetryAfter);
                        else
                            throw;
                    }
                     else 
                       throw;
                }
            }
        }
    }

This is bad code. The `catch (AggregateException aggregateException)` swallows the exception if the `aggregateException.InnerException` is not a `DocumentClientException`, potentially causing the `while(!queryDone)` to loop endlessly. — core, Mar 11 '16 at 17:36
@core While in theory you are correct, in practice I've had this code in production nearly a year now with a kill time set on the job - it has never reached this kill time, hence meaning your loop scenario can't be happening in practice. Regardless, it would have perhaps been useful to others for you to suggest the subtle fix / edit the answer than declare the full answer "bad code"! — BMac, Mar 15 '16 at 15:52
Should you really be using `Thread.Sleep` in async code? Isn't `Task.Delay` what you want? — jocull, Apr 03 '18 at 14:29

Exception: Microsoft.Azure.Documents.RequestRateTooLargeException while querying from DocumentDB

2 Answers2