12

I am using an Azure queue and have several different processes reading from the queue.
My system is built in a way that assumes each message is read only once.
This Microsoft article claims Azure queues have an at least once delivery guarantee which potentially means two processes can read the same message from the queue.
This StackOverflow thread claims that if I use GetMessage then the message becomes invisible to all other processes for the invisibility timeout.

Assuming I use GetMessage() and never exceed the message invisibility time before I DeleteMessage, can I assume I will get each message only once?

Community
  • 1
  • 1
Gilad Gat
  • 1,468
  • 2
  • 14
  • 19
  • I think you can assume that. – Richard Astbury Nov 28 '12 at 09:22
  • 2
    I asked the following question to my Microsoft Azure contact: "I’m relying heavily on queue’s and want to make sure that if I GetMessage() and then DeleteMessage() , no other process will get the same message from the queue. According to this [Microsoft article](http://msdn.microsoft.com/en-us/library/windowsazure/hh767287.aspx), queues have no At-Most-Once guarantee. Does this mean that two processes can read the same queue message and process it?" Response: Yes – this is exactly what it means. If you want that each message will be proceed exactly once you will have to use ServiceBus queue. – Gilad Gat Nov 28 '12 at 16:16

2 Answers2

10

I think there is a property in queue message named DequeueCount, which is the number of times this message has been dequeued. And it's maintained by queue service. I think you can use this property to identify whether your message had been read before.

https://learn.microsoft.com/en-us/dotnet/api/azure.storage.queues.models.queuemessage.dequeuecount?view=azure-dotnet

Tanya Branagan
  • 551
  • 1
  • 12
  • 21
Shaun Xu
  • 4,476
  • 2
  • 27
  • 41
  • What happens if processA checks DequeueCount and see it is 0. Then processA looses the CPU and processB checks DequeueCount and also finds it at 0. ProcessB then Dequeues the message and then processA wakes up again and also Dequeues the message. That way they both perform the business logic on the same queue message. Am I missing something here? – Gilad Gat Nov 29 '12 at 12:23
  • @Gilad - the `DequeueCount` is set by the service when the item is dequeued, so threading issues don't apply. So you could achieve at-most-once semantics by dropping any messages with `DequeueCount > 0`. But I don't think there's any way to get exactly-once semantics with queues. – Brian Reischl Nov 29 '12 at 15:26
  • That's correct as 'breischl' said, the `DequeueCount` is maintained by queue service. It will be +1 once a message was dequeued. The queue service itself achieve at-least-once, and if you checked the `DequeuCount > 0` then I think you achieve at-most-once. Only one thing, you'd better move the messages that dequeucount >0 to another 'poison message' instead of just delete them, as those messages probably means some background task was not finished yet. – Shaun Xu Nov 30 '12 at 04:48
  • 2
    Is that `DequeueCount` consistent in concurrency scenarios though? Could two threads dequeue the same message simultaneously and both see it as `DequeueCount: 0`? – Mike Asdf Jun 21 '13 at 20:05
  • 1
    You cannot rely on `DequeCount`. @MikeAsdf scenario is valid. In concurrent reading scenarios Microsoft makes not guarantee. You need to use Service Bus – Dave Hogan Sep 14 '16 at 11:14
8

No. The following can happen:

  • GetMessage()
  • Add some records in a database...
  • Generate some files...
  • DeleteMessage() -> Unexpected failure (process that crashes, instance that reboots, network connectivity issues, ...)

In this case your logic was executed without calling DeleteMessage. This means, once the invisibility timeout expires, the message will appear in the queue and be processed once again. You will need to make sure that your process is idempotent:

Idempotence is the property of certain operations in mathematics and computer science, that they can be applied multiple times without changing the result beyond the initial application.

An alternative solution would be to use Service Bus Queues with the ReceiveAndDelete mode (see this page under How to Receive Messages from a Queue). If you receive the message it will be marked as consumed and never appear again. This way you can be sure it is delivered At-Most-Once (see the comparison with Storage Queues here). But then again, if something happens while your are processing the message (ie: server crashes, ...), you could loose valuable information.

Update:

This will simulate an At-Most-Once in storage queues. The message can arrive multiple times via GetMessage, but will only be processed once by your business logic (with the risk that some of your business logic will never execute).

  • GetMessage()
  • DeleteMessage()
  • AddRecordsToDatabase()
  • GenerateFiles()
Sandrino Di Mattia
  • 24,739
  • 2
  • 60
  • 65
  • Assuming my process never crashes and DeleteMessage is **always** called within the invisibility period, can I assume no other process will get the same message? – Gilad Gat Nov 28 '12 at 09:51
  • Yes (some content to satisfy the minimum character count) – Sandrino Di Mattia Nov 28 '12 at 10:28
  • If this is the case I can do: queue.GetMessage(); immediately followed by queue.DeleteMessage(); in which case I have an **At-Most-Once** guarantee. If this is the case, why does Microsoft claim there is no **At-Most-Once** guarantee? – Gilad Gat Nov 28 '12 at 13:00
  • Yes. Even if the DeleteMessage fails, you won't have executed any business logic so it would be safe to process the message again the next time. – Sandrino Di Mattia Nov 28 '12 at 13:02
  • My point is as following: if what you're saying is true, why would Microsoft claim there is no At-Most-Once guarantee? – Gilad Gat Nov 28 '12 at 13:09
  • The call to `DeleteMessage()` can still fail. And the Guest OS could crash before calling `DeleteMessage()`. And the host OS can crash. And the hardware can fail. There's NO guarantee `DeleteMessage()` will be called or will succeed. Ergo... there's no at-most-once delivery guarantee. – David Makogon Nov 28 '12 at 13:26
  • True, but if he calls DeleteMessage immediately after the message arrives, you can assume **the business logic** executes at-most-once. (Note: the original comment did not use the term at-most-once) – Sandrino Di Mattia Nov 28 '12 at 13:30
  • I asked my MS Azure contact and her response was: "if you want that each message will be proceed exactly once you will have to use ServiceBus queue." Unfortunate, but as it's from Microsoft I guess it's official... – Gilad Gat Nov 28 '12 at 16:18
  • The answer from your Microsoft contact is incorrect. Sandrino is correct, and the logic described here gives "at most once" *processing* of messages. (This says nothing about delivery, just processing.) – user94559 Nov 28 '12 at 20:17
  • 1
    To follow up on Sandrino's example: let's say my queue has itemA and I have two processes. It is possible that both processes will perform GetMessage() and get ItemA, but one of them (the slower one) will fail when calling DeleteMessage() and therefor will not execute the business logic. Is this correct? – Gilad Gat Nov 28 '12 at 22:28