1

Hi i have a performance issue regarding Logic Apps and a Azure Service bus queue.

So i have a logic app looking like this:

enter image description here

(NOTE: The delay is there to simulate a bunch of connectors/actions that take about 2 seconds to run, also i'm using lock tokens and also session IDs to complete messages and Close sessions)

Its polling the service bus every second for high throughput with a peak-lock because my service bus queue uses sessions to enable FIFO Ordering in my flow. So what i'm doing is, sending about 2000 messages with different session-IDs and also some that are alike depending on the message as they relate to each other sometimes (hens the fifo) to my service bus.

Enabling the Logic app after all the messages are in the Queue, my Logic App only processes 10 messages at a time. In other words it only scales up to 10 runs simultaneously.

This is far to slow, as those 2000 messages takes about 15 minutes to process. That's about 2-4 seconds per run, 10 runs at a time depending on the last 2 connectors that can take up to 1 second each to complete the message and close the session.

What i have done to try to solve this by trying for example:

  • Running the "When one or more messages are received in queue (peak-lock)" instead with message count delivery bumped up to 175 (max value). This results still in only one message getting polled, guessing this has to do with the sessions enabled for the queue.

  • Turned on "High Throughput" on the logic app, resulting in only up to 10 instances running simultaneously anyway, sometimes even fewer.

  • Turned off (and even turned on with max value defined) the switch for "Concurrency Control" on the trigger for the logic app. Still same results.

  • Scaled up the service bus to standard tier, still no luck.

  • Tried to clone the same logic app up to 5 times running them in parallell, still same result.

and much more..

(Worth mentioning, the messages aren't bigger than 1,5kb - 1,8kb)

What am i missing? I can't think of any other configuration to change, or any other ideas using this combination of logic app and session enabled service bus queues to try next. Should i maybe even upgrade the service bus to premium? I didn't think it was worth it because i haven't met the limits for the standard tier yet.

If i forgot to post any useful data please tell me and i will edit the question.

EDIT:

2018-04-06 10:00

I how now tried a type of “Sequential Convoy” type of solution which yielded in even worse results:

I followed this guide with some modifications: Sequential Convoy Guide

The modification i did was putting a "for each" loop instead of a "do until". The thing with this solution was that it took the "get all messages in queue" action about 30 seconds to search for a message in the queue containing the defined session ID, which is way to much. This also resulted in a weird behavior which made the logic app to scale less parallel runs too.

enter image description here

2018-04-06 16:00

Another test i tried now was trying to skip the service bus to test the instance scaling of the logic app, and sure enough posting directly from the API to the Logic app by the Http Post trigger in logic app, the logic app runs a lot more instances that with the service bus in between.

Which means that it is the service bus queue limiting the amount of instances being run by the logic app.

I cant find anything that fixes this problem, even by trying do clone the logic app to multiple amounts pulling from the same queue...

Any ideas on this?

Community
  • 1
  • 1
H4p7ic
  • 1,669
  • 2
  • 32
  • 61
  • Have a look at this question, it is related to session https://stackoverflow.com/questions/49370957/logic-app-only-reading-a-few-service-bus-messages-before-timeout – Thomas Apr 06 '18 at 05:15
  • To increase the throughtput, I would suggest you to use eventgrid with servicebus and then call an http trigger logic app, dont know if it make sens for you ? – Thomas Apr 06 '18 at 05:24
  • I guess it makes sense, i think :P , but would this increase the Throughput really? I mean it feels (after reading of course) that the event grid would only be another addition to accumulated time in the integration flow so to speak. It feels like the bottle neck is something between the service bus and the logic app, but please right me if i'm wrong here i'm all ears by this state. And thanks for the answer. :) @Thomas – H4p7ic Apr 06 '18 at 08:46
  • it seems that the throughtput is pretty good: 10 millions of event per seconds (https://stackoverflow.com/a/48629117/4167200) So you push messages to servicebus, get it through eventgrid then from eventgrid call an http trigger logic app (with high throughput) and it should do the trick. you will be limited by the throughput of the logic app but it should fine, what do you reckon ? – Thomas Apr 07 '18 at 02:15
  • Does that sounds like an acceptable answer ? – Thomas Apr 08 '18 at 08:30
  • Thanks for the quick answers @Thomas, i'm just about to look into that right now, haven't had the time yet. But from what i can read it definitely sounds plausible. I'll definitely update the post with what i managed to get out of testing that solution. Thanks again! :) – H4p7ic Apr 08 '18 at 14:02
  • Hey @Thomas. So i've been reading up on this now, and i tried to get it to work, but when i try to create a Event Grid subscription of the type "webhook" i have to write down an subscription endpoint. is that the connection string for the service bus, cos that doesn't seem to work for some reason. – H4p7ic Apr 09 '18 at 15:20
  • @Thomas so i have been testing out event grid and from what my tests have yielded the event grid triggers the logic app per event. That is when the subscription gets the "messageS" that counts as one event. so its not an event per message which negates the whole purpose of the event grid. Also there is a delay of 1 minute before my logic app gets triggered by the event grid event. So as for now this does not seem to be the solution. – H4p7ic Apr 11 '18 at 13:27
  • Sorry to ear that. Hope you'll find a solution. – Thomas Apr 11 '18 at 20:38

2 Answers2

0

Few recommendations/inputs here, let us know if it helps.

  1. You can use "when one or more messages in queue peek-lock" with next available session instead of when one message trigger, the batch trigger will improve the performance.
  2. "Get session messages" action uses long polling and waits for 30 secs for messages to appear in a queue. If there are no messages in 30 secs the next actions will not get executed. If there were any messages available in the queue, they will be returned immediately.
  3. The sequential convoy implementation can be improved here. It can look for more messages for the same session. If you think there will be more messages available to process immediately, use do-until and keep on receiving more messages based on any of the conditions as mentioned in the blog post.
Andrew Williamson
  • 8,299
  • 3
  • 34
  • 62
Aprana Seth
  • 1
  • Hi @ApranaSeth, these are good guidelines but unfortunately ive tried them already. The thing is, the reason i need fifo is because i occasionally get messages that have to be kept in correct order, not always though. so the throughput on the messages with the same session isn't that high. Its a lot higher looking at it from an over all perspective so to speak. and that's why i need the logic app to scale alot more than it already does. – H4p7ic Apr 10 '18 at 07:28
0

I came across the same issue. One possible way in which I was able to get around it was by Cloning the Logic App. Multiple copies of Logic App helped in improving the consumption speed in peek-lock mode significantly.

Sasmit
  • 1