7

Sometimes there are some messages in Azure Queues that are not taken in charge by Azure Functions and also are not visible from StorageExplorer. These messages are created without any visibility delay.

Is there any way to know what do those messages contain, and why are they not processed by our Azure Functions?

StorageExplorer In the image you can see that we have a message in queue but it is not visible in the list and it is there from hours.

Antonio Salvati
  • 395
  • 3
  • 15
  • did you check out poison messages. deadletter queues https://learn.microsoft.com/en-us/azure/app-service-web/websites-dotnet-webjobs-sdk-storage-queues-how-to#poison – Aravind May 02 '17 at 08:54
  • 2
    Can you see if your function has executed (and failed) recently? That number indicates either 'expired' or 'invisible' messages. 'Invisible' messages should re-appear in the queue when the `InvisibilityTimeout` expires. – brettsam May 02 '17 at 14:36
  • 1
    I don't have an answer for you but I can say that this happens because some process read the message and therefore it is currently invisible until the invisibility timeout expires or the process manually releases it. The purpose of this mechanism is to stop multiple processes from processing the same message. – Ezekiel Victor Mar 04 '20 at 17:49

2 Answers2

2

The Azure Queue API currently has no way to check invisible messages.

There are several situations in which a message will become invisible:

  1. The message was added with an VisibilityTimeout in the Put Message request. The message will be invisible until this initial timeout expires.
  2. The message has been retrieved (dequeued). Whenever a message is retrieved it will be invisible for the duration of the VisibilityTimeout specified by the Get Messages request, or 30 seconds by default.
  3. The message has expired. Messages expire after 7 days by default, or after the MessageTTL specified in the Put Message request. Note: after a while these messages are automatically deleted, but until that point they are there as invisible message.

Use cases

Initial VisibilityTimeout

Messages are created with an initial VisibilityTimeout so that the message can be created now, but processed later (after the timeout expires), for whatever reason the creator has for wanting to delay this processing.

VisibilityTimeout on retrieving

The intended process for processing queue messages is:

  1. The application dequeues one or more messages, optionally specifying the next VisibilityTimeout. This timeout should be bigger than the time it takes to process the message(s).
  2. The application processes the message(s).
  3. The application deletes the messages. When the processing fails the message(s) are not deleted.
  4. Message(s) for which the process failed will become visible again as soon as their VisibilityTimeout expires, so that they can be re-tried. To prevent endless retries step 2. should start by checking the DequeueCount of the message: if it is bigger than the desired retry-count it should be deleted, instead of processed. It is good practice to copy such messages to a deadletter / poison queue (for example a queue with the original queue name plus a -poison suffix).

MessageTTL

By default messages have a time-to-live of 7 days. If the application processing cannot handle the amount of messages being added, a backlog could build up. Adjusting the TTL will determine what happens to such backlog. Alternatively the application could crash, so that the backlog would build up until the application would be started again.

Mark Lagendijk
  • 6,247
  • 2
  • 36
  • 24
0

It seems that the message is expired. The following steps could reproduce the issue, you could test it.

Add message with a short TTL

enter image description here

After the message has been expired

enter image description here

Fei Han
  • 26,415
  • 1
  • 30
  • 41