Azure Durable Function generates error 'Host thresholds exceeded: Connections' with CloudTable.ExecuteBatchAsync

Question

I am using Azure Durable Function.

I found out that it hits the connections' limit:

I have been able to identify the call that is responsible for hitting the limit which is:

public class RecordStorage : IRecordStorage
{
    private readonly CloudTableClient m_cloudTableClient;

    public RecordStorage()
    {
        CloudStorageAccount storageAccount = CloudStorageAccount.Parse(/* Connection string */);
        m_cloudTableClient = storageAccount.CreateCloudTableClient(new TableClientConfiguration());
        m_cloudTableClient.DefaultRequestOptions.RetryPolicy = Constants.Storage.RetryPolicy;
    }

    public async Task UpsertAsync(string listId, ICollection<RecordBatch> recordBatches)
    {
        CloudTable cloudTable = m_cloudTableClient.GetTableReference(listId);

        var tasks = new List<Task>();

        foreach(var recordBatch in recordBatches)
        {
             var tableBatchOperation = new TableBatchOperation();

             foreach(var batch in recordBatch.Records)
                  tableBatchOperation.Add(TableOperation.InsertOrReplace(new SomeTableEntity(***)));

             /* THIS IS THE CALL THAT APPEAR TO BE CAUSING THE SPIKE OF CONNECTION */
             tasks.Add(cloudTable.ExecuteBatchAsync(tableBatchOperation));
        }

        await Task.WhenAll(tasks);
    }
}

This method is called indirectly within an activity.

I do not understand why it hits the limit since I made sure to reuse the CloudTableClient instance. I also made sure the instance of RecordStorage is injected in the other classes as a Singleton:

public class Startup : FunctionsStartup
{
   public override void Configure(IFunctionsHostBuilder builder)
   {
        builder.Services.AddSingleton<IRecordStorage, RecordStorage>();
   }
}

Question

Why am I exceeding the connection limit if I keep reusing the same CloudTableClient instance?

It seems right, the only think i see its that your class does not implement the interface, how are you calling the UpsertAsync? Can you edit your question and add service that calls UpsertAsync? — Nacho Martínez-Aedo, Aug 07 '20 at 23:09
As per this [doc](https://learn.microsoft.com/en-us/azure/azure-functions/manage-connections#static-clients), you'd better creating a static CloudTableClient for re-use purpose. Could you please change your `CloudTableClient` to static like this `private static readonly CloudTableClient m_cloudTableClient;`? — Ivan Glasenberg, Aug 10 '20 at 09:18

score 0 · Accepted Answer · answered Aug 11 '20 at 20:40

0

This is embarrassing. I think I have just selected an incorrect aggregation method in the metric chart. I chose Max aggregation but I should have chosen Count instead. Here is the chart for the same period:

And the chart for the whole day:

answered Aug 11 '20 at 20:40

Kzryzstof

7,688
10
61
108

Azure Durable Function generates error 'Host thresholds exceeded: Connections' with CloudTable.ExecuteBatchAsync

1 Answers1