3

I have an Azure Function that calls a web service hosted on a VM. Because of this dependency it is important that no more than 8 instances of the Azure Function are ever running concurrently (the VM only has 8 cores and each call to the web service uses a single core). If more than 8 instances are spawned then the web service calls will start to backup, items from the queue that is triggering the Azure Function will start timing out, and messages will be dropped. However, I also want to utilize the available resources as much as possible to maximize the processing of the queue so I'd like there to always be 8 instances of the Azure function running whenever there are 8 or more items in the queue.

In order to accomplish the required throttling I have set the Azure Function plan to run on an App Service plan instead of a Consumption plan and I have set the following values in the host.json for the Azure Function service:

{
    "queues": {
        "batchSize": 1,
        "newBatchThreshold": 7
    }
}

Theoretically this makes it so that as long as 7 or fewer instances of the function are running then 1 more message will be dequeued until there are 8 messages being processed. I have this running right now and it seems to be working, but I can't find anywhere in the host.json documentation or the WebJobs SDK documentation that says whether or not it is okay for batchSize to be less than newBatchThreshold. I only know that the recommendation is for newBatchThreshold to be half of batchSize (which is clearly not what I am doing).

My question is: Is this configuration okay? Or is there a better way to accomplish my throttling goals?

sam2929
  • 479
  • 7
  • 14

1 Answers1

6

Yes, that is perfectly ok. Each flag has a precise (albeit kind of confusing) semantic, and any combination is valid and will honor the documented semantic.

Be aware that when using Consumption Azure Functions, you may end up getting scaled to multiple instances, each of which having those limits. If you want to avoid that try setting WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT to 1, per https://github.com/Azure/azure-webjobs-sdk-script/wiki/Configuration-Settings

David Ebbo
  • 42,443
  • 8
  • 103
  • 117
  • Thanks for the confirmation. I am aware that with a Consumption Azure Function it could end up getting scaled out to multiple instances, each with their own limits. That is why I chose to use an App Service plan, so that it would be limited to a single instance. However I didn't know about the `WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT` config setting so thanks for that tip! – sam2929 Sep 27 '17 at 18:34
  • 1
    @DavidEbbo `batchSize` is then confusing to me indeed. I thought that was a cap for max parallel executions per instance, so for value 1 I would not expect any parallel instances running. I thought `newBatchThreshold` would trigger prefetch, not immediate run. I guess I'm confusing it with Service Bus. – Mikhail Shilkov Sep 27 '17 at 19:37
  • @Mikhail yeah it's super confusing. As long as the currently executing count is less than the threshold, it keeps grabbing new batches – David Ebbo Sep 27 '17 at 19:46
  • @david If WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT is set to 1 and if you have a BatchSize > 1 then what actually happens? So I am limiting to a single instance yet I am grabbing a batch of X messages where X > 1 so what even is the meaning of a batch in that case? With a single instance I can’t process multiple items at once right? – Matthew Nov 27 '21 at 22:01