2

Even though I have a non-exclusive queue, I noticed that messages are delivered to consumers in round-robin manner based on when they connect. I am using JCSMP and set EndpointProperties to ACCESSTYPE_NONEXCLUSIVE.

Here is how I tested:

(1) Before starting any message consumers, I send the 1st batch of 20 messages to a queue.

(2) I start Consumer 1 which processes messages with a delay of 2 seconds each message.

(3) Next, I start Consumer 2 while Consumer 1 is consuming the 1st batch of messages. Even though Consumer 2 is running, but it is idling as messages from the 1st batch are not delivered to it.

(4) I send the 2nd batch of 20 messages. These messages from the 2nd batch are delivered in a round robin manner to Consumer 1 and Consumer 2.

(5) While Consumer 1 is processing messages from the 1st batch and 2nd batch, and while Consumer 2 is processing messages from the 2nd batch, I start Consumer 3. Although Consumer 3 is running but it is idling as no messages from 1st batch and 2nd batch are delivered to it.

(6) I send the 3rd batch of 20 messages. These messages from the 3rd batch are delivered in a round robin manner to Consumer 1, Consumer 2 and Consumer 3.

I was expecting that regardless when the messages are stored in the queue, the moment I start any consumers, messages should be delivered to them. This is important so that I can start more consumers to clear the backlog if messages are piled up in a queue. But, based on what I experimented, it does not seem to work. Did I miss out any settings somewhere?

Appreciate your advice. Thank you.

Koon Sang
  • 319
  • 3
  • 12

1 Answers1

2

At step 2 in your question, the queue will send a batch of messages to Consumer 1 before receiving an acknowledgement. The "Guaranteed Message Window Size" flow property of the consumer determines how many messages will be delivered, the default "Guaranteed Message Window Size" is 255 messages. The queue property called "Max Delivered Unacked Messages Per Flow" will also determine how many messages can be sent at a time from a queue to one consumer without receiving an acknowledgment, the default for this property is 10,000.

At step 3, the messages have already been sent to Consumer 1 and the queue is waiting for acknowledgements for these messages so Consumer 2 will not receive these messages.

The queue will send messages to consumers in batches by default to provide high throughput for applications that will acknowledge a message quickly. In your case where the application takes 2 seconds to process a message before acknowledging it, it would be beneficial to edit the two properties mentioned above to allow for one messages to be delivered to a consumer at a time.

You can set both the "Guaranteed Message Window Size" property on the consumer flow and the "Max Delivered Unacked Messages Per Flow" property on the queue to 1. This will allow for the queue to only deliver one message before it receives an acknowledgement, so when a new consumer binds to the non-exclusive queue, it will receive the next message.

Alexandra Masse
  • 1,277
  • 1
  • 7
  • 11
  • Hi Alex, it seems that setting "Max Delivered Unacked Messages Per Flow" property on the queue to 1 could already make sure "the queue to only deliver one message before it receives an acknowledgment". Why do we need to set "Guaranteed Message Window Size" to 1 in this situation? – flyisland Dec 08 '17 at 04:30
  • 1
    Hi @flyisland, the "Guaranteed Message Window Size" determines how many messages can be in flight before they are acked by the API, as opposed to a client acknowledgement. By default, this API ack will be sent when 60% of the window is reached or after a 1 second timer. When the "Max Delivered Unacked Messages Per Flow" value is set to a value lower than 60% of the "Guaranteed Message Window Size", the API ack will not be sent until the timer expires, thus slowing down the rate that messages are sent to the router. – Alexandra Masse Dec 14 '17 at 19:59
  • This behavior is improved in recent versions of the API but we still always recommend that the "Guaranteed Message Window Size" should not exceed the "Max Delivered Unacked Messages Per Flow" value in order to achieve the best performance. – Alexandra Masse Dec 14 '17 at 20:02