Logic app completing a message in a service bus topic subscription with sessions does not release the lock

Question

I have an topic subscription on a service bus in Azure with a lock duration of 30 seconds and sessions enabled.

I'm using a logic app to take messages of the topic using peak-lock because I care about the order messages are processed. I want to ensure that all messages with the same session id are processed in the order they are added to the bus i.e. FIFO.

Below is a screenshot of a simple logic app which is triggered when a message arrives (peek-lock with 5 second polling), it then waits for 15 seconds before completing the message.

When I load 10 messages on to the topic, the first instance of the logic app is triggered as you would expect and completes in roughly 15 seconds, however the second logic app doesn't start for another 15 seconds i.e. 30 seconds in total after the first logic app started (the length of the lock duration).

Everything I have read suggests that completing a message should release the lock immediately but this doesn't appear to be the case. Am I doing something wrong or is this the way it is meant to work?

Answer 1

Everything I have read suggests that completing a message should release the lock immediately but this doesn't appear to be the case. Am I doing something wrong or is this the way it is meant to work?

LockDuration instructs a queue (subscription is a queue after all) to keep messages invisible from competing consumers to ensure the current processing node can handle it within the time frame defined by LockDuration. Renew lock mechanism ensures that the lock on the currently being processed message is extended to allow longer than LockDuration processing time if needed so. An important thing to remember, lock renewal is not guaranteed to be successful.

Scenario 1:

However, I have tried replicating your workflow with a small change in Topic Subscription Lock Duration as 2 minutes (to see if LockDuration property influences the LA trigger) and provided the SessionID manually. Below is my test result.

I have loaded the first 10 messages into the topic.

From the above images, you could see the second instance of the Logic App Trigger initiated right after completing the first instance.

Scenario 2: In this case, I have maintained the same configuration but changed the SessionId to "NextAvailable". Now, The Logic App trigger awaits the entire lock duration (2 min and even more sometimes) and triggers the next instance.

In conclusion, if you having the same session Id for your messages and configuring the Session ID as "Next available" in your LA UI, then it locks the message for the entire LockDuration time frame. If you want to avoid this, then you need to "Close a session in a queue" at the client-side. This will immediately release the lock and process the next instance immediately.

Answer 2

The combination to use in order to achieve your desired outcome is as follows, assuming you are using a topic and not a queue, although it possible to achieve same with queue, just use the equivalent queue activities:

1. Use When a message is received in a topic subscription (peek-lock) trigger to pull message from subscription, use 'Next available' as the Session Id

2. Initialise a variable e.g. WorkflowStatus and set to nothing, this will signal the next step to break out of loop

3. Following step 2, create a parallel branch, Use Logic App Activity Renew lock on the session in a topic subscription inside an Until Logic App Activity together with a Delay activity

4. Use the following to break out Until loop: length(variables('WorkflowStatus')) is greater than 0

5. Ensure the delay is at least 5 seconds less than your topic's max lock duration

6. Create a try, catch, finally scope in your main branch

7. When your Logic App workflow completes, first, signal the Until activity to stop renewing your lock

8. Signal by setting WorkflowStatus variable to Success inside the end of try scope and Failure inside the beginning of catch scope

9. Use Complete the message in a topic subscription activity in the finally scope if WorkflowStatus is success

10. Use Close a session in the topic activity in the finally scope if WorkflowStatus is success

11. Use Dead-letter the message in a topic subscription activity in the finally scope if WorkflowStatus is failure

12. Best practice to terminate the Logic App workflow as either succeeded or failed based on value of WorkflowStatus

13. Best practice to offer up the Dead-letter the message in a topic subscription reason for dead-lettering as the Workflow Error Message extracted from catch scope

Here is an example:

Improvements:

1. You could include another Until activity in the try block to wait for more messages to arrive in the locked session for a configurable duration and fetch messages by session Id that the trigger provided you.
2. Abandon the message if the work in the try scope fails for a reason that warrants another go e.g. a remote endpoint is unreachable even after retry exhaustion

This example uses the following tools:

• Core Tools Version: 4.0.5030
• Function Runtime Version: 4.15.2.201
• Azure Logic Apps (Standard) v1.0.46 Visual Studio Code Extension
• Azurite v3.22.0 Visual Studio Code Extension

For more information on how to extract the error message in the catch scope see Approach 1 at the following link

Logic App Exception Handling With The Result Function

For more information on how to use an Until activity to fetch more messages from the topic for a configurable amount of time, see the sequential convoy pattern at the following link

While there are more messages for the session in the queue

You only really need to do this if you anticipate 'large volumes' of messages per Session Id.

Answer 3

When you close the session, it assumes there are no more messages on it and then you lose them all. How do I avoid losing messages and still processing them asap and not wait for the lock duration to finish? I have a Sessionful receiver, next available, I complete the message as soon as I receive it and then I close the session. Meanwhile, a new message on that session arrived, and because I Closed the session, that message is lost.