0

Background

I have a set of logic apps that each call a set function apps which are run in parallel.

Each logic app is triggered to start at a certain time during the night with all staggered an hour apart.

The Azure functions are written using the async pattern and call external APIs.

Problem

Sometimes the logic apps will run fine and complete their execution in a normal time period, and can do so for two or three days in a row.

However sometimes they will take hours or days forcing me to cancel their run.

enter image description here

Can any body shed any light on this might be happening?

Notes

  1. I'm using the latest nuget packages of the durable functions extension
  2. When debugging the functions always complete in a timely fashion
  3. I have noticed that the functions sometimes get stuck at pending.
Dan Cundy
  • 2,649
  • 2
  • 38
  • 65
  • 2
    First, if you look in app insights are your functions throwing errors resulting in them taking long to complete? Second, test deploying to a new logic app and function instance since it seems a very peculiar scenario. – Murray Foxcroft Jan 25 '19 at 09:58
  • You could click the Runs history , check you logic flow which step stuck. – George Chen Jan 28 '19 at 05:50
  • @MurrayFoxcroft No execeptions are being thrown. I do however have a failing dependency call to the storage table which i'm investigating. – Dan Cundy Jan 28 '19 at 10:08
  • @MurrayFoxcroft Yep, its the function app that is getting stuck, all other logic apps steps are successful. – Dan Cundy Jan 28 '19 at 10:10
  • Good luck. Sort out all the errors and make sure you are not swallowing any exceptions try { } catch {} type scenarios and you should be closer to a solution. – Murray Foxcroft Jan 28 '19 at 10:47
  • 1
    Hi Dan, I'd like to take a closer look at what's going on with your Durable Functions. I've opened up an [issue in our GitHub repo](https://github.com/Azure/azure-functions-durable-extension/issues/592) to track this; would you mind hopping over there and providing the information under the "Investigative Information" header so we can investigate? Thanks! – Katy Shimizu Jan 28 '19 at 23:16

1 Answers1

2

It appears you have at least two function apps that are configured with the same storage account and task hub name:

  • AzureConsumptionXXX
  • AzureComputeXXX

This causes the two function apps to steal messages from each other. If functions in one app do not exist in the other app, then it's very possible for orchestrations to get stuck in a Pending state like this.

The simplest way to mitigate this is to give each function app a unique task hub name. Please see the Task Hubs documentation for more information: https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-task-hubs.

Chris Gillum
  • 14,526
  • 5
  • 48
  • 61