5

Just out of curiosity: What exactly happens behind the scenes when we call and await ServiceBusReceiver.ReceiveMessageAsync() or ServiceBusReceiver.ReceiveMessageAsync(TimeSpan.FromMinutes(10))?

Does

a.) the ServiceBusReceiver (long-)poll Azure Service Bus or does

b.) Azure Service Bus send somehow push notifications to the ServiceBusReceiver?

I tried to look into the source code, but didn't get far, because some class InnerReceiver is used that I couldn't find in the code base.

Joerg
  • 790
  • 2
  • 10
  • 23
  • Ran to the same problem... waiting to see if anyone has an answer for this. Maybe you can test it using network screener and look at the network communication with azure... – Ethan.S Mar 01 '23 at 09:53

3 Answers3

5

FINAL EDIT

On the ServiceBus Sdk level it is a pull communication model, just to better handle confirmation of incoming messages.

But behind the scenes azure service bus sdk uses Advanced Message Queuing Protocol 1.0 which doesn't long poll, it uses AmqpLinks on the same AmqpConnection to send data from a sender to a receiver, and the receiver saves the incoming messages to an in-memory buffer on receiving them.

You can read more about it here or See this video series from Microsoft about the AMQP 1.0 protocol.

So it seems that the answer to "What exactly happens behind the scenes" is neither A nor B, Messages are being pushed to the ReceivingAmqpLink (to an in-memory buffer), which then the ServiceBusReceiver pulls from.

I hope this answer will close this discussion, and everyone will be happy :)

The source codes:

Azure.Messaging.ServiceBus

Azure.Amqp

ServiceBusReceiver wraps the AmqpReceiver

wrapping the ReceiveMessagesAsync of the AmqpReceiver

Creating the ReceivingAmqpLink in the constructor of the AmqpReceiver

Opening the ReceivingAmqpLink on the same connection if possible

Caching the messages on the ReceivingAmqpLink's buffer

Pulling the messages from the buffer into the ReceiveAsyncResult

The public async function that creates the ReceiveAsyncResult

Calling the that "public async function" on AmqpReceiver.ReceiveMessagesAsync

Ethan.S
  • 385
  • 3
  • 13
  • 1
    Actually, I am not sure about this. Just now, I've come across [this section](https://learn.microsoft.com/en-us/azure/architecture/guide/technology-choices/messaging#pull-model) by accident: `A consumer of a Service Bus queue constantly polls Service Bus to check if new messages are available. The client SDKs and Azure Functions trigger for Service Bus abstract that model. When a new message is available, the consumer's callback is invoked and the message is sent to the consumer.` So I guess, it's polling after all. – Joerg Mar 01 '23 at 12:03
  • 1
    Ok so after your comment I did some more reading and it's not completely true. The `AmqpReceiver` implementation doesn't use polling in it's root. It accumulate the messages sent through the AMQP connection to a inner in memory queue, end expose a thread safe very high level in-memory polling system for the user to use. The `ServiceBusReceiver` is just a wrapper class for the `AmqpReceiver` and expose the "polling system" to use. – Ethan.S Mar 01 '23 at 16:44
  • So you're saying messages are pushed to the client and then locally polled from memory? That would be nice. – Joerg Mar 01 '23 at 18:50
  • It looks like it :) – Ethan.S Mar 01 '23 at 19:31
  • 1
    Maybe your answer could be even better if you provided a deep-link to the source code pin-pointing where and how exactly `Azure.Messaging.ServiceBus` is referencing/calling `Azure.Amqp`. The provided links only point to the repository roots. Not criticizing, just optimizing. ;-) – Joerg Mar 02 '23 at 07:22
  • Just did hope it will be helpful :) – Ethan.S Mar 02 '23 at 08:52
  • 1
    Ethen's answer is correct; the client API pattern is pull-based for receivers and push-based for the processor. Under the covers, credits are managed on a persistent AMQP link to control message flow. When prefetch is not used, credits are only placed on the link when a receiver asks for messages (which is continually for the processor and when ReceiveAsync is called for the receiver) – Jesse Squire Mar 02 '23 at 15:15
  • 1
    Nice source spelunking, by the way. :) – Jesse Squire Mar 02 '23 at 15:26
2

I think @Ethen.S answer is not correct. Moreover neither (a) nor (b) from original question are true:

You might want to take a look at Azure.Messaging.ServiceBus.ServiceBusProcessor (use StartProcessingAsync method as an entry point for investigation) source to better understand the dichotomy there. It exposes an event that you subscribe to and handle the messages "reactively", but it still does a sequence of the ReceiveMessageAsync for you. What might look "push" model from the consumer perspective might still rely on "poll" model behind the scenes.

Please also note that there is a concept of Message Prefetching which can optimise some of the consumption scenarios, but it has its caveats to be considered as well (see https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-prefetch?tabs=dotnet#why-is-prefetch-not-the-default-option). It is still the same polling strategy, but with an additional client-level buffer on top.

UPDATE. I dug deeper into that to capture what happens on wire.

I've used sslsplit in order to make MITM attack to capture TLS-protected AMQP connection decrypted data. Test application setup is simple:

  • Receiver is doing ReceiveMessageAsync in a loop
  • After some time sender sends a message (two times)

In summary, this is indeed neither "poll" nor "push" as definitions are rather vague. What we can say for sure is that:

  • it uses persistent TCP connection
  • receival is controlled by the Client side using "Link Credit" (via the "flow" performative)
  • when client is idle waiting for messages once in about 30 seconds empty AMQP message is sent
  • Service Bus has mechanism to notify the receiver efficiently if there are enough "Link Credit"

I've put test code and test harness here: https://github.com/gubenkoved/amqp-test.

Below is the capture results, I've colored sender/receiver TCP streams as red/green.

capture

Eugene D. Gubenkov
  • 5,127
  • 6
  • 39
  • 71
  • This is only the ServiceBus sdk interface and what I called the "Polling System". If you will dig down to the AMQP 1.0 implementation you will see that all messages sent through an AMQP link are being sent from the sender and saved to an in-memory buffer. After the "Polling System" asked for those messages and finish processing them, the receiver (ours `ReceivingAmqpLink`) will send back a confirmation that we successfully processed the messages. – Ethan.S Mar 02 '23 at 00:39
  • I fully edited my answer to be more precise, behind the scenes it doesn't long poll the messages from service bus. – Ethan.S Mar 02 '23 at 01:18
  • 1
    To be pedantic, it's not inherently push or pull, it's a streaming model where message flow is controlled by the client via managing credits on an AMQP link. How eagerly messages are streamed in depends on whether prefetch is enabled. Whether or not you consider it "polling" really comes down to how you define the term. – Jesse Squire Mar 02 '23 at 15:25
  • @JesseSquire I agree with you. The protocol has attributes of both approaches. Thank you for this comment! I will look into expanding my answer later on this one. – Eugene D. Gubenkov Mar 03 '23 at 09:20
  • 1
    @JesseSquire I've added packet capture results to show what is really happening on wire – Eugene D. Gubenkov Mar 04 '23 at 00:30
  • 1
    LGTM. WireShark does a nice job of capturing individual AMQP frames. For completeness, it may also make sense to link to the discussion of AMQP + Service Bus, which itself links out to the official AMQP spec, in case folks are interested in the low-level details. https://learn.microsoft.com/azure/service-bus-messaging/service-bus-amqp-overview – Jesse Squire Mar 05 '23 at 16:03
1

According to this section of the Azure Application Architecture Guide, Azure Service Bus consumers use a poll model when talking to Azure Service Bus:

Pull model

A consumer of a Service Bus queue constantly polls Service Bus to check if new messages are available. The client SDKs and Azure Functions trigger for Service Bus abstract that model. When a new message is available, the consumer's callback is invoked and the message is sent to the consumer.

Since ServiceBusReceiver is such a consumer, it probably uses polling.

Joerg
  • 790
  • 2
  • 10
  • 23
  • I would like if you can show me in the source code where exactly the polling occurs. – Ethan.S Mar 01 '23 at 17:01
  • 1
    I can't. I only quoted the official documentation above which I tend to trust. However, I am still not 100 % percent convinced myself. Hence, I have not accepted either of our two answers, yet. I am still hoping for an answer from some Microsoft person. – Joerg Mar 01 '23 at 18:48
  • 1
    @Ethen.S I have just seen the edits in your answer. That convinces me. – Joerg Mar 01 '23 at 18:54
  • I am with you on that, Microsoft's documentation is a shit show. – Ethan.S Mar 01 '23 at 19:29
  • 1
    I wouldn't be that harsh. Overall I find it rather good compared to other documentation. :) – Joerg Mar 01 '23 at 20:03
  • 1
    No idea who owns that doc, but it's incorrect - or at least inaccurate, depending on your definition of "polling". Regardless of client, a persistent AMQP connection and link are established, allowing messages to stream in. The client controls this flow by adding credits to the link which controls how much data the service streams down. For a receiver, messages stream eagerly when prefetch is enable; without prefetch only when ReceiveAsync is called. – Jesse Squire Mar 02 '23 at 15:18