4

I have a Azure Function that handles SharePoint operations. Due to throttling I only want 10 concurrent functions running at the time and always 10 functions concurrently. I was considering to use functions with an Azure Service Bus Queue. Is there any build in the Azure platform to achieve this? Service Bus is not a requirement, so if other hubs or queues are better for this. I have looked in the Azure UI and havent found anything on the service bus side or function.

Test and observations:

I created a test with no success. I have created and deployed a functions that sleeps for 20 secs and then write a document to Cosmos DB:

[FunctionName("SleepFunction")]
public static void Run([ServiceBusTrigger("provision", AccessRights.Manage, Connection = "AzureWebJobsServiceBus")]string myQueueItem, TraceWriter log)
{
    System.Threading.Thread.Sleep(20000);
    log.Info($"C# ServiceBus queue trigger function processed message: {myQueueItem}");

    string EndpointUrl = "https://kk-db-governance-engine.documents.azure.com:443/";
    string PrimaryKey = "xx";
    var docClient = new DocumentClient(new Uri(EndpointUrl), PrimaryKey);
    var result = docClient.CreateDocumentAsync(UriFactory.CreateDocumentCollectionUri("kk-governance", "test-function-concurrency"), new Run());

}

Functions app.settings: enter image description here

If I add 10 messages to the queue, 10 documents are added to the database concurrently. I would expect that only 3 at the would be added with 20 secs delayed. Setting the WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT in the app settings does not seem seem to work. Any suggestions?

I also tried the maxConcurrentCalls in the host.json file:

{
  "IsEncrypted": false,
  "Values": {
    "AzureWebJobsServiceBus": "xxx",
    "AzureWebJobsStorage": "xx",
    "AzureWebJobsDashboard": "xx"
  },
  "version": "2.0",
  "extensions": {
    "serviceBus": {
      "maxConcurrentCalls": "3"
    }
  }
}

As you can see on screenshot below I use consumption plan: enter image description here

Answer: I got following working:

"serviceBus": { "maxConcurrentCalls": 3
}

Thomas Segato
  • 4,567
  • 11
  • 55
  • 104
  • When you mean 10 functions, you mean you want to max a maximum of ten concurrent calls to SP ??? – Thomas Dec 16 '18 at 21:35
  • Yes exactly. I have one single function, and I want 10 concurrent instances of that single function running at all time. So if I add 10000 messages to the queue, 10 instances will run concurrently until all messages has been processed. – Thomas Segato Dec 16 '18 at 21:42
  • As per this post https://github.com/Azure/azure-functions-host/issues/912#issuecomment-419608830, you can set this setting `WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT` to `1`. It will force having only one instance of your function app running. And then you can change the max concurrent call to 10 in the `host.json file`. I think both storage queues and servicebus queues should work fine. – Thomas Dec 16 '18 at 21:49
  • Thanks, highly appreciated. I will give it a test and get back. – Thomas Segato Dec 16 '18 at 21:59
  • Do you by any chance have a sample host.json? There is tons of articles describing that this should be set, but I havent found a single sample. I tried: { "WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT": 3 } but this throws an error. – Thomas Segato Dec 16 '18 at 22:06
  • 1
    WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT should be app setting. just put this in the app settings in the portal – Thomas Dec 17 '18 at 02:20
  • 2
    Also if you want to configure the `host.json` file, check out the documentation: https://learn.microsoft.com/en-us/azure/azure-functions/functions-host-json – Thomas Dec 17 '18 at 04:18
  • Hi @Thomas I tried your suggestion, but didn't seem to work. I have updated post with findings. – Thomas Segato Dec 17 '18 at 16:06
  • Hope this thread gives some insight - https://github.com/Azure/azure-functions-host/issues/2667 – Baskar Rao Dec 17 '18 at 18:13
  • Thanks @Baskar. Is the conclusion then you cannot currently have x numbers of functions running without developing your own system? – Thomas Segato Dec 17 '18 at 18:46
  • @ThomasSegato, the `WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT` control the number of function apps that are spin up. If you want to change the max concurrent call, you need modify the host.json and set the max concurrent call also . – Thomas Dec 17 '18 at 20:24
  • check the `maxConcurrentCalls` https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-service-bus#host-json – Thomas Dec 17 '18 at 20:25
  • @ThomasSegato I am not sure, but I would wait for some official confirmation from Functions team. – Baskar Rao Dec 17 '18 at 20:28
  • OK, I will see if I can get an answer from them (the Microsoft team). – Thomas Segato Dec 17 '18 at 22:52

2 Answers2

8

An answer can be found here: Add Singleton support for Functions to ensure only one function running at a time

  1. If you're running on the consumption plan: To ensure you won't scale out to more than one instance, set this app setting:
    • WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT = 1

  2. To allow only X concurrent calls in this instance, you can specify this setting in the host.json file:

    { "version": "2.0", "extensions": { "serviceBus": { "maxConcurrentCalls": 1 } } }

Fka
  • 6,044
  • 5
  • 42
  • 60
Thomas
  • 24,234
  • 6
  • 81
  • 125
  • 1
    Thanks @Thomas. I tried your suggestion, with no luck. I added finding to my main post. – Thomas Segato Dec 18 '18 at 09:50
  • Have you tried with both settings: WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT set to 1 andmaxConcurrentCalls set to 3 ? – Thomas Dec 18 '18 at 10:47
  • 1
    Jeps, both andmaxConcurrentCalls and is set to 3. I think it is an issue in the product see https://learn.microsoft.com/en-us/azure/azure-functions/functions-host-json. I have just created a support case to get this clarified. – Thomas Segato Dec 18 '18 at 12:03
2

For those who find a way to this question, the actual host.json structure has changed since the accepted answer.

To allow only X concurrent calls in this instance, you can specify this setting in the host.json is now:

"extensions": {
    "serviceBus": {
      "prefetchCount": 1,
      "batchOptions": {
        "maxMessageCount": 1
      }
    }
  }
William Ratte
  • 41
  • 3