Spring JMSListener Merging the messages

Question

Implemented Spring JMS with Listener with concurrency '5' threads ,

<jms:listener-container container-type="default" concurrency="5-10" connection-factory="cachingConnectionFactory"   >
    <jms:listener destination="TEST.FOO" ref="messageListener" />
 </jms:listener-container>

When I drop 5 messages , 5 threads are listening and I can able to read the messages .

My question is how to merge all the 5 messages ,Is it possible to write some builder where builder can wait for sometime ,so when any message received within that time ,so I can merge all the messages?

Code:

    long startTime = 0;
                if (messageCount == 0) {
                    startTime = System.currentTimeMillis();
                }    
messageCount ++;
            if (messageCount < batchSize && (startTime > (System.currentTimeMillis() - maximumBatchWaitTime))) { // Or if some batch timelimit say

                        line += stringMessage;
                        reached = true; // this is volatile variable ,messageCount also volatile variable
                    }

                    System.out.println(line);

                    try {
                        if (reached) {
                            messageCount = 0;
                            line = "";
                            execService.createExecFile(line);

                        }
                    } catch (final Exception e) {
                        // TODO Auto-generated catch block
                        e.printStackTrace();
                    }

Regards, Rj

can you explain more what you mean by **merge all the 5 messages** — Hassen Bennour, Jan 20 '17 at 10:01

Hassen Bennour · Accepted Answer · 2017-01-23T13:14:50.913

your messageListener is a singleton, so all threads will uses the same instance, like this you can synchronize a call to a method to append messages to a new line as a field on that instance.

UPDATE 1

use org.springframework.jms.core.JmsTemplate.receive()

import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.MessageListener;
import javax.jms.TextMessage;

import org.apache.activemq.command.ActiveMQTextMessage;

public class DefaultMessageListener implements MessageListener {
    private volatile String line = "";
    private volatile int messageCount;
    private int batchSize;
    private volatile long startTime = 0;
    private long maximumBatchWaitTime;

    Thread thread = new Thread(new Runnable() {
        @Override
        public void run() {
            while (true) {
                try {
                    Thread.sleep(200);
                } catch (InterruptedException e) {
                }
                if (messageCount > 0 && (startTime > (System.currentTimeMillis() - maximumBatchWaitTime))) {
                    createExecFile();
                }
            }
        }
    });

    {
        thread.start();
    }

    @Override
    public synchronized void onMessage(Message message) {
        if (messageCount == 0) {
            startTime = System.currentTimeMillis();
        }
        try {
            messageCount++;
            line += ((TextMessage) message).getText();
            System.out.println(line);
            // (startTime > (System.currentTimeMillis() - maximumBatchWaitTime))
            // why to do this ?? not needed i think
            if (messageCount == batchSize) {
                createExecFile();
            }
        } catch (final Exception e) {
            e.printStackTrace();
        }
    }

    private void createExecFile() {
        try {
            execService.createExecFile(line);
        } catch (final Exception e) {
            e.printStackTrace();
        }
        messageCount = 0;
        line = "";
    }
}

UPDATE 2

import javax.jms.Message;
import javax.jms.TextMessage;

import org.apache.activemq.ActiveMQSession;
import org.apache.activemq.command.ActiveMQMessage;
import org.springframework.jms.core.JmsTemplate;

public class DefaultMessageListener {
    private volatile String line = "";
    private volatile int messageCount;
    private long maximumBatchWaitTime;
    JmsTemplate jmsTemplate;

    public void getMessages() {
        try {
            // configure bean jmsTemplate like this on definition
            jmsTemplate.setSessionAcknowledgeMode(ActiveMQSession.INDIVIDUAL_ACKNOWLEDGE);
            jmsTemplate.setReceiveTimeout(maximumBatchWaitTime);
            //
            Message message = jmsTemplate.receive();
            if (message != null) {
                messageCount++;
                line += ((TextMessage) message).getText();
            }
            System.out.println(line);
            if (messageCount > 0) {
                createExecFile();
            }
            message.acknowledge();
        } catch (final Exception e) {
            e.printStackTrace();
        }
    }

    private void createExecFile() {
        try {
            execService.createExecFile(line);
        } catch (final Exception e) {
            e.printStackTrace();
        }
        messageCount = 0;
        line = "";
    }
}

INDIVIDUAL_ACKNOWLEDGE : unacknowledged messages are being re-delivered when the consumer connects again. examples http://alvinalexander.com/java/jwarehouse/activemq/activemq-core/src/test/java/org/apache/activemq/JMSIndividualAckTest.java.shtml

UPDATE 3

import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.Session;
import javax.jms.TextMessage;

import org.apache.activemq.ActiveMQSession;
import org.springframework.jms.core.JmsTemplate;

public class DefaultMessageListener {
    private volatile String line = "";
    private volatile int messageCount;
    private long maximumBatchWaitTime;
    JmsTemplate jmsTemplate;
    private ActiveMQSession session;

    public void getMessages() throws JMSException {
        try {
            // configure bean jmsTemplate like this on definition
            jmsTemplate.setSessionAcknowledgeMode(ActiveMQSession.CLIENT_ACKNOWLEDGE);
            jmsTemplate.setReceiveTimeout(maximumBatchWaitTime);
            //
            Message message = jmsTemplate.receive();
            if (message != null) {
                messageCount++;
                line += ((TextMessage) message).getText();
            }
            System.out.println(line);
            if (messageCount > 0) {
                createExecFile();
            }
        } catch (final Exception e) {
            e.printStackTrace();
            session.recover();
            messageCount = 0;
            line = "";
        }
    }

    private void createExecFile() {
        execService.createExecFile(line);
        session.acknowledge();
        messageCount = 0;
        line = "";
    }
}

Comments are not for extended discussion; this conversation has been [moved to chat](http://chat.stackoverflow.com/rooms/133874/discussion-on-answer-by-hassen-bennour-spring-jmslistener-merging-the-messages). — Bhargav Rao, Jan 24 '17 at 09:28
@Hassen Bennour, I also have a need to make a List of messages that is received from Queue, Can you please suggest If your answer mentioned under "update 1" will be better for this. https://stackoverflow.com/questions/55107244/receiving-multiple-messages-from-mq-asynchronously/55422434?noredirect=1#comment97582655_55422434 — Alagammal P, Mar 30 '19 at 22:14
@AlagammalP you can test it yep, or use the BatchMessageListenerContainer. — Hassen Bennour, Apr 01 '19 at 07:32
@Hassen Bennour, I will try the one under "UPDATE 1" . I couldnt get the good examples of BatchMessageListenerContainer to see how it is consuming multiple messages. — Alagammal P, Apr 01 '19 at 09:22
@AlagammalP take a look here https://stackoverflow.com/a/28973075/6506229 — Hassen Bennour, Apr 01 '19 at 09:29
@Hassen Bennour, I think this line "if (messageCount == batchSize) { createExecFile();} " inside onMessage() is not required If we don't want to block the method for next thread with new message. — Alagammal P, Apr 01 '19 at 15:55
@AlagammalP keep in mind that the list of messages is shared by all threads, so when a thread persist the messages to DB and iterate over the list you need to block another threads to prevent ConcurrentModificationException, commit to DB, acknowledge messages one by one or commit the JMS session and clear the list. i think that the BatchMessageListenerContainer is the best solution to not have to rewrite what exists — Hassen Bennour, Apr 02 '19 at 07:11
@Hassen Bennour, I thought If change the condition inside run() method to if (messageCount == batchsize || (startTime > (System.currentTimeMillis() - maximumBatchWaitTime))) then it can be handed over to another thread which can do DB updates independently list of messages will also be cleared. I think JMS message will be committed as soon as it comes out of onMessage(). — Alagammal P, Apr 02 '19 at 08:54
@AlagammalP possible yes but if you don't manage ACK manually messages added to the list are acked to the broker and if you have a persistency problem or app crashes ? you have to handle that — Hassen Bennour, Apr 02 '19 at 10:15
@Hassen Bennour, I tried the answer in UPDATE 1 section and it is not working. I created a new post https://stackoverflow.com/questions/55498472/consuming-messages-from-mq-and-merging-in-spring-jms — Alagammal P, Apr 03 '19 at 15:28

Spring JMSListener Merging the messages

1 Answers1