-1

I am building an Azure function that retrieves delta updates from Service Bus and should create/update documents based on that in Cosmos DB.

Currently I am only using the output binding to achieve this, but I am running into 429 errors when I have a lot of updates in the queue, even with 20.000 RU/s.

It looks like I need to throttle the amount of data coming from Bus -> Azure Function -> Cosmos DB. What is the "best practice" way to approach this?

I can only find limited information on this topic.

David Makogon
  • 69,407
  • 21
  • 141
  • 189
mp1990
  • 9,699
  • 3
  • 13
  • 17
  • Does this answer your question? [Automatic retry to CosmosDb output binding](https://stackoverflow.com/questions/50695921/automatic-retry-to-cosmosdb-output-binding) – Ecstasy Jan 31 '22 at 10:33
  • [How to use Azure functions to process high throughput messages](https://www.freecodecamp.org/news/how-to-use-azure-functions-to-process-high-throughput-messages-996d05d4ab23/) – Ecstasy Jan 31 '22 at 10:34
  • RU/sec are spread across physical partitions (e.g. if you have 5 physical partitions, you're going to see 4K/sec, when allocating 20,000/sec). How many physical partitions do you have? As for "best practice" - there's just no single right answer to that. – David Makogon Jan 31 '22 at 13:00
  • @DavidMakogon I was just wondering what the best practice is to handle the 429 errors coming from the Cosmos DB SDK. I'm not sure what physical partitions have to do with that. Maybe i failed to describe my question. – mp1990 Feb 02 '22 at 13:40
  • @DeepDave-MT thank you for the links, the output bindings just seem to add a layer on top of the already established retry functionality of Service Bus. I guess for now I will let the Exception bubble to Service Bus and let it handle retrying the 429 errors. – mp1990 Feb 02 '22 at 13:41

1 Answers1

0

Throttling problem "429 - too many requests" error can be solved using Azure Logic App service. As you have asked for the best approach, this is one approach which can solve your issue with less efforts.

Below are some key points you need to take care when using Logic App:

  • Limit the number of logic app instances that can run at the same time.

  • Enable high throughput mode.

  • Disable array debatching ("split on") behavior in triggers.

  • Refactor actions into smaller logic apps.

Refer: Handle throttling problems (429 - "Too many requests" errors) in Azure Logic Apps

Utkarsh Pal
  • 4,079
  • 1
  • 5
  • 14