5

I understand from the Microsoft docs that during the first Peek() operation, any one of the available message brokers respond and send their oldest message. Then on subsequent Peek() operation, we can traverse across the partitions to peek every message with increased sequence number.

My question is, during the very first Peek() operation, I will get a message from any of the first responded partitions. Is there a guarantee that I can peek all the messages from the queue?

In a much simpler way, there are three Partitions: Partition "A" has 10 messages with sequence number from 1 to 10. Partition "B" has 10 messages with sequence number from 11 to 20. Partition "C" has 10 messages with sequence number from 21 to 30.

Now if i perform Peek() operation, if Partition "B" responds first, the first message that I'll get is a message with sequence number 11. Next peek operation will look for a message with incremented sequence number. Won't I miss out messages from Partition "A" which has sequence numbers 1-10 which peek operation can never reach since it always searches for the incremented sequence number?

UPDATE

QueueClient queueClient = messagingFactory.CreateQueueClient("QueueName", ReceiveMode.PeekLock);
BrokeredMessage message = null;
while (iteration < messageCount)
{
  message = queueClient.Peek(); // According to docs, Peeks the oldest message from any responding broker, and next iterations peek the message with incremented sequence number
  if (message == null)
    break;
  Console.WriteLine(message.SequenceNumber);
  iteration++;
}

Is there a guarantee that I can browse all the messages of a partitioned queue using the snippet above?

Deepak
  • 2,660
  • 2
  • 8
  • 23

2 Answers2

0

There is no guarantee that the returned message is the oldest one across all partitions.

It therefore depends which message broker responds first, and the oldest message from that partition will be shown. There is no general rule as to which partition will respond first in your example, but it is guaranteed that the oldest message from that partition is displayed first.

If you want to retrieve the messages by sequence number, use the overloaded Peek method Peek(Sequencenumber), see: https://learn.microsoft.com/en-us/azure/service-bus-messaging/message-browsing

Casper Dijkstra
  • 1,615
  • 10
  • 37
  • What if i need to peek messages from the lowest to highest sequence number in order? – Deepak Oct 26 '20 at 14:33
  • If you know the lowest and highest sequence numbers, it's doable. Let me check if it's also possible if you don't know these numbers – Casper Dijkstra Oct 26 '20 at 14:45
  • Peek starting at zero? – Triynko Dec 01 '20 at 05:39
  • @Triynko Well that seems pretty easy to make an ordered retrieval though, but its mentioned nowhere and that's the issue using this Peek(0) thing :P I get an ordered way but still maybe I cannot reproduce a scenario where it doesn't – Deepak Dec 25 '20 at 04:04
0

For partitioned entities, the sequence number is issued relative to the partition.

[...]

The SequenceNumber value is a unique 64-bit integer assigned to a message as it is accepted and stored by the broker and functions as its internal identifier. For partitioned entities, the topmost 16 bits reflect the partition identifier. (https://learn.microsoft.com/en-us/azure/service-bus-messaging/message-sequencing)

So you cannot compare sequence numbers across partitions to see which one is older.

As an example, I just created a partitioned queue and put a couple of messages into two partitions (in Order):

1. Partition 1, SequenceNumber 61924494876344321
2. Partition 2, SequenceNumber 28991922601197569
3. Partition 1, SequenceNumber 61924494876344322
4. Partition 1, SequenceNumber 61924494876344323
5. Partition 2, SequenceNumber 28991922601197570

Browse/Peek messages: Available only in the older WindowsAzure.ServiceBus library. PeekBatch does not always return the number of messages specified in the MessageCount property. There are two common reasons for this behavior. One reason is that the aggregated size of the collection of messages exceeds the maximum size of 256 KB. Another reason is that if the queue or topic has the EnablePartitioning property set to true, a partition may not have enough messages to complete the requested number of messages. In general, if an application wants to receive a specific number of messages, it should call PeekBatch repeatedly until it gets that number of messages, or there are no more messages to peek. (https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-partitioning, Emphasis added)

As such, you should be able to repeatedly call Peek / PeekBatch to eventually get all the messages. At least, if you use the official SDKs.

Alex AIT
  • 17,361
  • 3
  • 36
  • 73
  • 2
    During the first Peek operation, is there no guarantee that the message with the lowest sequence number across the partitions is peeked first? If so, how can I browser all the messages of a queue? Because Peek method works by incrementing the sequence number. So if i get a message from Partition 2 during the first peek operation (say if partition 2 responded first), will I miss out messages from Partition 1 in your example? – Deepak Oct 27 '20 at 03:29
  • 1
    Seriously. Mohan's question is relevant. Does the QueueClient also keep track of which partition it's on and which partitions it hasn't visited yet? For example, if partition 5 responds first, will it finish enumeration of partition 5, and then begin enumeration at some other random partition, and so on, until all partitions have been visited? – Triynko Dec 01 '20 at 05:38