4

I've noticed that looping (using foreach) in Azure Logic Apps is very slow. As an example I've looped over a http result containing 8.000 JSON nodes (altogether 1.6MB data) and it took around 6 minutes to process which is an extremely long time compared with doing it in a .Net application. This is how the test was done: Test logic app, in this test the JSON data comes from a blob

For me it's a common task to loop through a result set, should it be this slow? Do any of you know a better way to parse and loop data in Logic Apps?

TobiasJ
  • 111
  • 1
  • 7
  • Perhaps the best practice is to use a Function App for this? Doing the same parsing in a Function App took around 1 second. – TobiasJ Mar 16 '18 at 15:47

1 Answers1

2

The reason it is slow is because the concurrency for loops is default of 20 and max of 50. The best way around this is to use a slightly different pattern, where you are debatching to another logic app. For example you'll have your one logic app that generates the array that you want to loop through, then create another logic app that accepts an http request as a trigger. On the second logic app under settings enable the spliton function. Now put your 'work' in the second logic app, then go back to the first and make the last step to send the array to the second logic app. When using this pattern the first logic app will initialize a run for each item in the array, they will all run concurrently, rather than limited to the concurrency of the for each loop. Here is some more detail in the documentation about this function. https://learn.microsoft.com/en-us/azure/logic-apps/logic-apps-workflow-actions-triggers#split-on-debatch

Phil
  • 86
  • 5
  • Nice answer, interesting approach :-) – Thomas Mar 17 '18 at 04:55
  • Thanks Phil. Very useful. What I actually is using now in the "Data operations - Select" action because what I want to do is to transform the JSON nodes to another format, and this action turned out to be very fast (although it's supposed to do a for each implicitly). – TobiasJ Mar 20 '18 at 14:54
  • @Phil i am facing a similar issue. My logic app processes json data as header and items. The items takes a long time in For Each loop step. Will Split on help. ? Should i have two logic app , one the main and aother just to loop through?. – Winona Feb 17 '21 at 16:49
  • @phil This was a brilliant approach. I was able to reduce the duration of my logic app from 4.58 Hours to 10.33 Minutes just by nesting the for loop process in another logic app. Thank you sir. – A23149577 Feb 16 '22 at 02:22