Error in MQTT Apache Camel route causes application to stop

Question

I'm using Apache Camel and Spring (not Spring Boot) with xml.

I have my camel-context.xml configuration file with a sample route from MQTT server to JMS server and viceversa, just to dispatch the messages.

Here is my camel-context.xml:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:context="http://www.springframework.org/schema/context"
    xmlns:util="http://www.springframework.org/schema/util"
    xmlns:camel="http://camel.apache.org/schema/spring"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
                        http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
                        http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd
                        http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd">

    <bean id="quadtreeProcessor" class="es.gateway.router.QuadtreeProcessor" />

    <camelContext id="camelContext" xmlns="http://camel.apache.org/schema/spring">
        <route id="quadTreeConsumerProducer">
            <from uri="jms:topic:T_ETSI_PRODUCER"/>
            <process ref="quadtreeProcessor"/>
            <to uri="mqtt:quadtree?host=tcp://localhost:1883&amp;publishTopicName=${header.publishTopicName}"/>
        </route>

        <route id="quadTreeConsumerRoute">
            <from uri="mqtt:quadtree?host=tcp://localhost:1883&amp;subscribeTopicName=CONSUMER/DENM/#"/>
            <to uri="jms:topic:T_ETSI_CONSUMER"/>
        </route>
    </camelContext>

    <bean id="jmsConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
        <property name="brokerURL" value="tcp://localhost:10011"/>
    </bean>
    <bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent" />

And here is my Main.class:

package es.conncar.main;

import org.apache.camel.RuntimeCamelException;
import org.apache.camel.main.MainListener;
import org.apache.camel.main.MainListenerSupport;
import org.apache.camel.main.MainSupport;
import org.apache.log4j.Logger;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;

import es.gateway.config.Config;

public class Main {

    final static Logger logger = Logger.getLogger(Main.class);

    private static boolean exit = false;

    private org.apache.camel.spring.Main camelMain;

    public static void main(String[] args) throws Exception {
        try {
            ApplicationContext springContext = new ClassPathXmlApplicationContext("app-context.xml");
            logger.info("Spring context initialized");

            Config config = (Config) springContext.getBean("config");

            Main main = new Main();
            main.boot();
        } catch (Exception e) {
            logger.error("Unknown error in main", e);
        }
    }

    public void boot() throws Exception {
        camelMain = new org.apache.camel.spring.Main();
        camelMain.addMainListener((MainListener) new Events());
        camelMain.setApplicationContextUri("camel-context.xml"); 
        logger.info("Starting Camel. Use ctrl + c to terminate the JVM.\n");
        camelMain.run();
    }

    public static class Events extends MainListenerSupport {

        @Override
        public void afterStart(MainSupport main) {
            logger.info("Camel is now started!");
        }

        @Override
        public void beforeStop(MainSupport main) {
            logger.info("Camel is now being stopped!");
        }
    }
}

The problem ocurrs when running the same route and configuration within my Main class, which dies after a few seconds because of a RuntimeCamelException.

The exception trace is:

Camels starts ok

[30/11/2018 08:37:08][INFO ][es.gateway.main.Main] - Spring context initialized
[30/11/2018 08:37:08][INFO ][es.gateway.main.Main] - Starting Camel. Use ctrl + c to terminate the JVM.

After 10 seconds

[30/11/2018 08:37:21][ERROR][es.gateway.main.Main] - Unknown error in Camel
    org.apache.camel.RuntimeCamelException: java.util.concurrent.TimeoutException
        at org.apache.camel.util.ObjectHelper.wrapRuntimeCamelException(ObjectHelper.java:1830)
        at org.apache.camel.spring.SpringCamelContext.start(SpringCamelContext.java:136)
        at org.apache.camel.spring.CamelContextFactoryBean.start(CamelContextFactoryBean.java:369)
        at org.apache.camel.spring.CamelContextFactoryBean.onApplicationEvent(CamelContextFactoryBean.java:416)
        at org.apache.camel.spring.CamelContextFactoryBean.onApplicationEvent(CamelContextFactoryBean.java:94)
        at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
        at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
        at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
        at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:393)
        at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:347)
        at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:883)
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:546)
        at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:139)
        at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:93)
        at org.apache.camel.spring.Main.createDefaultApplicationContext(Main.java:222)
        at org.apache.camel.spring.Main.doStart(Main.java:154)
        at org.apache.camel.support.ServiceSupport.start(ServiceSupport.java:61)
        at org.apache.camel.main.MainSupport.run(MainSupport.java:170)
        at es.gateway.main.Main.boot(Main.java:105)
        at es.gateway.main.Main.main(Main.java:77)
    Caused by: java.util.concurrent.TimeoutException
        at org.fusesource.mqtt.client.Promise.await(Promise.java:83)
        at org.apache.camel.component.mqtt.MQTTEndpoint.connect(MQTTEndpoint.java:348)
        at org.apache.camel.component.mqtt.MQTTConsumer.doStart(MQTTConsumer.java:38)
        at org.apache.camel.support.ServiceSupport.start(ServiceSupport.java:61)
        at org.apache.camel.impl.DefaultCamelContext.startService(DefaultCamelContext.java:3705)
        at org.apache.camel.impl.DefaultCamelContext.doStartOrResumeRouteConsumers(DefaultCamelContext.java:4023)
        at org.apache.camel.impl.DefaultCamelContext.doStartRouteConsumers(DefaultCamelContext.java:3958)
        at org.apache.camel.impl.DefaultCamelContext.safelyStartRouteServices(DefaultCamelContext.java:3878)
        at org.apache.camel.impl.DefaultCamelContext.doStartOrResumeRoutes(DefaultCamelContext.java:3642)
        at org.apache.camel.impl.DefaultCamelContext.doStartCamel(DefaultCamelContext.java:3494)
        at org.apache.camel.impl.DefaultCamelContext.access$000(DefaultCamelContext.java:209)
        at org.apache.camel.impl.DefaultCamelContext$2.call(DefaultCamelContext.java:3253)
        at org.apache.camel.impl.DefaultCamelContext$2.call(DefaultCamelContext.java:3249)
        at org.apache.camel.impl.DefaultCamelContext.doWithDefinedClassLoader(DefaultCamelContext.java:3272)
        at org.apache.camel.impl.DefaultCamelContext.doStart(DefaultCamelContext.java:3249)
        at org.apache.camel.support.ServiceSupport.start(ServiceSupport.java:61)
        at org.apache.camel.impl.DefaultCamelContext.start(DefaultCamelContext.java:3165)
        at org.apache.camel.spring.SpringCamelContext.start(SpringCamelContext.java:133)
        ... 18 more

It looks like the route throws a timeout when not able to connect to the MQTT broker (that's ok) but this exception is not caught within the route but in the context - Main class (that's not ok).

I've followed this cookbook:

http://camel.apache.org/running-camel-standalone-and-have-it-keep-running.html

with camel-spring JAR in the org.apache.camel.spring.Main class.

I also checked the chapter 13 in the book Apache Camel in Action but I did not find a solution to this. It seems I'm starting and configuring camel context just fine.

Anybody has experience this? Is there a way to keep routes working and Main programm alive when RuntimeExceptions ocurrs in the routes? I hope so!

Thansk in advance!

EDIT: I have found several topics talking about this. The solution seems to be activating the Supervising Controller, but I can't find a way to do it without Spring Boot (I'm using plain Spring with mixed xml and annotations config). Anybody could help?

How to start camel even if the MQTT server is not reachable

Spring Context shutting down camel routes when activemq connection is lost

EDIT2: I have checked that the problem is only with MQTT. The JMS connection works correctly and is able to go on even when JMS broker is down and handle handle reconnections gracefully, while the MQTT connection not.

EDIT3: Paho works fine but plain MQTT not. I've tested the same code with Paho in xml config (uri) and it behaves as expected, similar to JMS. The route throws exceptions but keep going, it doesn't raise the exception to the context which is what causes the application to stop. Maybe I'm missing options in the MQTT client?

Edit the question to show the actual exception & stack trace and your actual code (maybe just the bit with the try/catch block if it's too long) — hardillb, Nov 29 '18 at 20:21
Of course, I will provide the code and config tonorrow. Thanks! — Dani P., Nov 29 '18 at 22:51
Stacktrace shows that exception comes from mqtt. Could you try all these settings with amqp and see if that fixes it? — YetAnotherBot, Nov 30 '18 at 12:51
Also, if you are on an IDE try debugging [this](https://github.com/apache/camel/blob/master/components/camel-mqtt/src/main/java/org/apache/camel/component/mqtt/MQTTEndpoint.java#L348) line and see what is the problem and what is the configuration for timeout seonds it is using. — YetAnotherBot, Nov 30 '18 at 12:56
Thanks all for your answers. I've tested this same code using Paho instead of plain MQTT in xml config (uri) and it works fine, so it must be something related to the particular implementation of the MQTT client. To me the problem is that the exception should be caught within the route and not raised yo the context. — Dani P., Nov 30 '18 at 17:00
Catch the exception using the `onexception` XML dsl structure and set handled to true. — Namphibian, Dec 04 '18 at 01:17
For some reason, I am not getting the notifications for this thread. Do try creating a `deadletterchannel` through camel for failure and experiment when things get desperate. Also, I believe `onexception` might help too. — YetAnotherBot, Dec 04 '18 at 16:08
I tried using onexception also, but the MQTT error was not handled. Definitely, it seems like a problem with MQTT client implementation. My workarround was just using Paho Client. Thanks for your help. — Dani P., Dec 05 '18 at 20:42
@Daniel Perales You may want to raise a request on their github page if you feel, your configurations open a bug in the code. Also, tag people in the chat so they can receive notification about the new messages. — YetAnotherBot, Dec 06 '18 at 05:53

score 2 · Accepted Answer · answered Jan 03 '19 at 12:26

I suggest to use camel-paho as Eclipse Paho is more active maintained than the older FuseSource MQTT client library that camel-mqtt uses.

That said, the problem with camel-mqtt component is that it requires a working connection when it startup. You can configure its reconnect options to set various delays etc.

There is an alternative route startup mechanism that allows to startup routes with a background thread that monitors the routes and can deal with retries etc. Its the SupervisingRouteController, note there is a few things still to implement around its JMX management capabilities, but otherwise ought to be fine. And it lacks more proper documentation. We are considering making it a more prominent or as default in Camel 3.

There is an example of it here: https://github.com/apache/camel/tree/master/examples/camel-example-spring-boot-supervising-route-controller

Thanks Claus, that piece of information is exactly what I was looking for. And good job with Camel :-) — Dani P., Jan 09 '19 at 20:02

Error in MQTT Apache Camel route causes application to stop

1 Answers1