3

I am writing a JMS consumer using spring-jms to consume message from the ActiveMQ Artemis broker. I am investigating the behaviour of 3 properties: concurrentConsumers, maxConcurrentConsumers, and idleConsumerLimit.

Expected behaviour is when load in queue increase consumer should scale up to maxConcurrentConsumers and when load decreases consumer should scale down to idleConsumerLimit.

But this is not happening with spring-jms and ActiveMQ Artemis. When there is no message in the queue consumer count is not coming down instead it retaining all the maxConcurrentConsumers.

I am using spring DMLC container

    <bean id="jmsContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
        <property name="connectionFactory" ref="connectionFactory" />
        <property name="destinationName" value="TestingInQueue" />
        <property name="messageListener" ref="messageListener" />
        <property name="concurrentConsumers" value="1"/>
        <property name="maxConcurrentConsumers" value="20"/>
        <property name="idleConsumerLimit" value="5"/>
    </bean>

Complete context XML:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:amq="http://activemq.apache.org/schema/core" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
  http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">

    <bean id="jndiTemplate" class="org.springframework.jndi.JndiTemplate">
        <property name="environment">
            <props>
                <prop key="java.naming.factory.initial">org.apache.activemq.artemis.jndi.ActiveMQInitialContextFactory</prop>
                <prop key="java.naming.provider.url">tcp://localhost:61616</prop>
                <prop key="java.naming.security.principal">admin</prop>
                <prop key="java.naming.security.credentials">admin</prop>
            </props>
        </property>
    </bean>

    <bean id="connectionFactory" class="org.springframework.jndi.JndiObjectFactoryBean">
        <property name="jndiTemplate" ref="jndiTemplate"/>
        <property name="jndiName" value="ConnectionFactory"/>
    </bean>

    <bean id="destinationQueue" class="org.apache.activemq.artemis.jms.client.ActiveMQQueue">
        <constructor-arg index="0" value="TestingInQueue" />
    </bean>

    <bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate">
        <property name="connectionFactory" ref="connectionFactory" />
        <property name="defaultDestination" ref="destinationQueue" />
    </bean>

    <bean id="messageListener" class="com.practise.SampleListener">
        <property name="jmsTemplate" ref="jmsTemplate" />
        <property name="queue" ref="destinationQueue" />
    </bean>

    <bean id="jmsContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
        <property name="connectionFactory" ref="connectionFactory" />
        <property name="destinationName" value="TestingInQueue" />
        <property name="messageListener" ref="messageListener" />
        <property name="concurrentConsumers" value="1"/>
        <property name="maxConcurrentConsumers" value="20"/>
        <property name="idleConsumerLimit" value="5"/>
    </bean>
</beans>
Justin Bertram
  • 29,372
  • 4
  • 21
  • 43
sujin
  • 2,813
  • 2
  • 21
  • 33

1 Answers1

4

Note:

  • maxConcurrentConsumers : Your maximum consumers under load
  • concurrentConsumers : You will eventually get here if you are idle for some time. (The time it takes to reach here depends on next 3 params)

The following 3 are to avoid consumers drastically scaling up and down between maxConcurrentConsumers and concurrentConsumers without any time delay.

  • idleConsumerLimit
  • maxMessagesPerTask
  • idleTaskExecutionLimit

Example

If you start with heavy load, you will have 20 (maxConcurrentConsumers) consumers. Then if you make the queue empty, it will scale down to 5 (idleConsumerLimit) . Then it will wait idleTaskExecutionLimit attempts to be reached and then scale down to 1 (concurrentConsumers). idleTaskExecutionLimit is number of fetch attempts which resulted in empty fetch where each fetch take receiveTimeout (default 1000ms) time unit.

Issue:

Default value of maxMessagesPerTask is -1 and idleConsumerLimit is not taking effect if maxMessagesPerTask is not greater than zero

Solution:

When specifying idleConsumerLimit, specify maxMessagesPerTask too.

Specify a number of 10 to 100 messages to balance between rather long-lived and rather short-lived tasks here for maxMessagesPerTask

Reference

Kavithakaran Kanapathippillai
  • 7,883
  • 1
  • 11
  • 29
  • Thanks for the Answer. Setting maxMessagesPerTask value worked. After setting this value consumer count came down to concurrentConsumers count but not to idleConsumerLimit. What is the effect of idleConsumerLimit in this configuration? – sujin Jun 25 '20 at 12:31
  • Even with ``maxMessagesPerTask``, ``idleConsumerLimit`` not taking effect, but consumer count able to shrink down to ``concurrentConsumers `` count. – sujin Jun 25 '20 at 12:49
  • I set ``maxMessagesPerTask`` as 10 for testing, then consumer count came down to 1 that is ``concurrentConsumer`` count. If i start the application with empty queue then 1 consumer created. I expect while scaling down consumer count has to be 5 that is ``idleConsumerLimit`` – sujin Jun 25 '20 at 13:10
  • Yes It was ignored – sujin Jun 25 '20 at 13:13
  • with ```idleTaskExecutionLimit``` property after making queue empty after sometime consumer gradually shrinking to 1. my question here is how long will wait from ```idleTaskExecutionLimit ``` to ```idleConsumerLimit```(5) ? Can you help pls – sujin Jun 25 '20 at 14:22
  • how long will wait from idleTaskExecutionLimit to idleConsumerLimit(5) ? Can you help pls - Yes. It will wait 5 times before it shrinks to math.max(idleConsumerLimit, concurrentConsumer) – krupalpatel86 Jan 20 '22 at 14:19