In an Azure Web Job, when is an item in the Azure queue removed?

Question

I have created a triggered web job using the Azure queue. For brevity, my function looks like this:

    public static void ProcessQueueMessage([QueueTrigger("myqueue")] string message, TextWriter log)
    {
        log.WriteLine(message);
        System.Threading.Thread.Sleep(10000);
    }

I have another function that adds items to the queue for processing. However, I want to check the last xx messages to determine if an item already exists in the queue (with the same message). Something like this:

while (true)
{
    var myMessageId = "98uasd98fusad8";
    CloudQueue queue = queueClient.GetQueueReference("myqueue");
    queue.CreateIfNotExists();
    var maxMessagesToCheck = 99;
    var oldMessages = queue.GetMessages(maxMessagesToCheck, null);
    if (!oldMessages.Any(x => x.AsString == myMessageId))
    {
        CloudQueueMessage message = new CloudQueueMessage(myMessageId);
        queue.AddMessage(message);
    }
}

However, I'm just not sure when an item is "popped" off the queue. Does it get popped off after processing is complete and the ProcessQueueMessage function returns (without error)? Or, does it get popped off right as the processing starts?

If the items are popped off after the processing completes, then my second function above will work fine. However, if items are popped off as soon as the process starts, then I will have to find another solution.

This might be a more concise way of asking the question:

    public static void ProcessQueueMessage([QueueTrigger("myqueue")] string message, TextWriter log)
    {
        //IS THE ITEM IN THE QUEUE POPPED OFF HERE?
        log.WriteLine(message);
        System.Threading.Thread.Sleep(10000);
    } //OR IS THE ITEM IN THE QUEUE POPPED OFF HERE?

Don't have time to answer, but it's hidden from the queue for the duration of `ProcessQueueMessage` and then popped at the end. If there's an exception thrown, it becomes visible again for retry, depending on your settings. — ProgrammingLlama, Jun 02 '18 at 03:13

score 0 · Answer 1 · answered Jun 07 '18 at 18:19

According to Microsoft doc

When a message is locked, other clients receiving from the same queue or subscription can take on locks and retrieve the next available messages not under active lock. When the lock on a message is explicitly released or when the lock expires, the message pops back up at or near the front of the retrieval order for redelivery.

For that reason your solution will not work, let's say you have 10 messages in queue, and the first message is currently being processed by your first function, therefore when the checking function runs it can only peek messages after it, let's say it is peeking message 2,3,4 in a batch, same as the first function it needs to put lock for the three messages, so when the first function finishes message 1, it can only pick up message 5, which has't checked.

In an Azure Web Job, when is an item in the Azure queue removed?

1 Answers1