5

I am using ActiveMQ 5.8 with wildcard consumers configured in camel route.

I am using default ActiveMQ configuration, so I have defaults as below

prefetch = 1
dispatch policy= Round Robin

Now I start a consumer jvm with 5 consumers each for 2 queues. both the queue has same type of message and same number of messages.

Consumers are doing nothing but printing the message (so no db blocking or slow consumer issue)

EDIT I have set preFetch to 1 for each of the queue

What I observe is one of the queue getting drained faster than other.

What I expect is both the queue getting drained at equal pace, kind of load balance.

One surprising observation is Though activemq webconsole shows 5 consumers for each of those queues

enter image description here

When I debug my consumer, I see only 5 threads / consumers from camel flow for a wildcard queue *.processQueue enter image description here

What will be cause of above behavior? How do I make sure that all the queue drain at equal pace?

Did anyone has experience to share on writting custom dispatch policy or overriding defaults of activemq?

Dhananjay
  • 3,903
  • 2
  • 29
  • 44

2 Answers2

2

I was able to find a reference to this behavior

Message distribution in case of wildcard queue consumers is random.

http://activemq.2283324.n4.nabble.com/Wildcard-and-message-distribution-td2346132.html#a2346133

Though this can be tuned by setting appropriate prefetch size.

After trial & error, I arrived at following formula, to have fair distribution across the consumers and all the queue getting de-queued at almost same pace.

prefetch = number of wildcard consumers
Dhananjay
  • 3,903
  • 2
  • 29
  • 44
0

It's probably wrong to compare the rate the queues are consumed. The load balancing typically happens between consumers. So, the idea is that each of the five consumers on the first queue would get rather even load (given they are connected to the same broker).

However, I think you might want to double check your load test setup. It rarely gives predictable results when running broker and consumers on the same machine for instance.

Petter Nordlander
  • 22,053
  • 5
  • 50
  • 84
  • I tested with running activemq server and consumers on other machine, still I see same behavior. both the queue consumers were on a single machine, does that make a difference? – Dhananjay Aug 19 '14 at 10:55
  • I do not have insight on how ActiveMQ is dispatching to the consumer under the covers, but I have a case where I have a few hundred queues that can be drained with a single route using ">". My observations are the same as what you've seen. My reasons for this are not for efficiency though. What are you attempting to solve that couldn't be solved by just using a single queue? – Erik Williams Sep 11 '14 at 18:06