Camel Route Losing Message on restart in camel rabbitmq

Question

I am using camel-rabbitmq. Here is my route defination

camelContext.addRoutes(new RouteBuilder() {
        @Override
        public void configure() throws Exception {
            from("rabbitmq:TEST?queue=TEST&concurrentConsumers=5")
            .routeId("jms")
            .autoStartup(false)
            .throttle(10)
            .asyncDelayed()
            .log("Consuming message ${body} to ${header.deliveryAddress}")
            .process(new Processor() {

                @Override
                public void process(Exchange exchange) throws Exception {
                        System.out.println(atomicLong.decrementAndGet());
                }
            })

            ;
        }
    });

When I push 500 messages to this queue , when stop and start route all message on channel will be lost ,wonder where they are going.

If I configure same route with &autoAck=false it is working properly but losing performance. Why camel not offering same behavior with and without autoAck.

score 0 · Accepted Answer · answered Apr 05 '20 at 16:51

I managed my problem doing following change in rabbitmqconsumer of camel-rabbitmq

    public void handleCancelOk(String consumerTag) {
        // no work to do
        log.info("Received cancelOk signal on the rabbitMQ channel");
        **downLatch.countDown();**
    }

  @Override
    protected void doStop() throws Exception {
        if (channel == null) {
            return;
        }
        this.requeueChannel=openChannel(consumer.getConnection());
         if (tag != null && isChannelOpen()) {
            channel.basicCancel(tag);
        }
        stopping=true;
         downLatch.await();         

        try {
            lock.acquire();
            if (isChannelOpen()) {
                channel.close();
            }
        } catch (TimeoutException e) {
            log.error("Timeout occured");
            throw e;
        } catch (InterruptedException e1) {
            log.error("Thread Interrupted!");
        } finally {
            lock.release();
        }


    }

By doing this camel route will for message to consumed and avoided message loss.

You are just replicating the behaviour of autoack = false by rejecting the message via shutdown hook. But just curious, what performance gain did you observe? — Sunand Padmanabhan, Apr 06 '20 at 07:16
Getting huge performance gain, also autoAck=false also cause duplicate message,consumer hang. — S Shukla, Apr 06 '20 at 13:49

score 0 · Answer 2 · answered Apr 09 '20 at 07:30

0

You need to check rabbitmq consumer prefetch count consumer prefetch I think By default consumer picks all the messages in queue to its memory buffers. If you set the prefetch count to 1, consumer will acknowledge messages one by one. All the other unacknowledged will be present in the queue in ready state. Waiting to be picked up, after the consumer completes it task on the previous message picked.

answered Apr 09 '20 at 07:30

Devansh Arora

31
7

You can check apache camel rabbitmq doc to get how to use prefetch options in a camel rabbit route https://camel.apache.org/components/latest/rabbitmq-component.html – Devansh Arora Apr 09 '20 at 07:34
Prefetch count works with manual ACK. Manual ACK will drop performance. And problem was with auto ac on – S Shukla Apr 09 '20 at 10:24
yes manual ack would impact performance. When i first faced this situation, loosing messages on restart was not an option for me. So i used manual ack and increased the number of concurrent consumers. – Devansh Arora Apr 14 '20 at 11:52

Camel Route Losing Message on restart in camel rabbitmq

2 Answers2