camel-elasticsearch5 component is not working

Question

My project consume messaged from activemq and store in to elasticsearch 2.4 database with technology karaf 4.0.8, Apache camel (2.18.1), camel-elasticsearch component. I am trying to upgrade to elasticsearch 5 database and that required changes in client side. I found that new camel component camel-elasticsearch5 is there for that with apache-camel 2.19.0. I upgrade the camel. after that it looks it is not working as expected. 1. the camel commands like camel:context-list is not working. 2. The route is not working like messaged are not consumed from activemq.

Additionally there is a warning in the log.

2017-07-18 06:57:27,144 | INFO  | nt Dispatcher: 1 | ElasticsearchProducer            | 21 - org.apache.camel.camel-elasticsearch5 - 2.19.0 | Connecting to the ElasticSearch cluster: elasticsearch
2017-07-18 06:57:27,145 | WARN  | -SNAPSHOT/deploy | BlueprintEventDispatcher         | 83 - org.apache.aries.blueprint.core - 1.7.1 | Listener caused an exception, will be ignored
java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: Could not initialize class org.elasticsearch.transport.client.PreBuiltTransportClient
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)[:1.8.0_121]
        at java.util.concurrent.FutureTask.get(FutureTask.java:192)[:1.8.0_121]
        at java.util.concurrent.AbstractExecutorService.doInvokeAny(AbstractExecutorService.java:193)[:1.8.0_121]
        at java.util.concurrent.AbstractExecutorService.invokeAny(AbstractExecutorService.java:225)[:1.8.0_121]
        at org.apache.aries.blueprint.utils.threading.ScheduledExecutorServiceWrapper$4.call(ScheduledExecutorServiceWrapper.java:184)[83:org.apache.aries.blueprint.core:1.7.1]
        at org.apache.aries.blueprint.utils.threading.ScheduledExecutorServiceWrapper$15.call(ScheduledExecutorServiceWrapper.java:452)[83:org.apache.aries.blueprint.core:1.7.1]
        at org.apache.aries.blueprint.utils.threading.RWLock.runReadOperation(RWLock.java:35)[83:org.apache.aries.blueprint.core:1.7.1]
        at org.apache.aries.blueprint.utils.threading.ScheduledExecutorServiceWrapper.runUnlessShutdown(ScheduledExecutorServiceWrapper.java:447)[83:org.apache.aries.blueprint.core:1.7.1]
        at org.apache.aries.blueprint.utils.threading.ScheduledExecutorServiceWrapper.invokeAny(ScheduledExecutorServiceWrapper.java:178)[83:org.apache.aries.blueprint.core:1.7.1]
        at org.apache.aries.blueprint.container.BlueprintEventDispatcher.callListener(BlueprintEventDispatcher.java:199)[83:org.apache.aries.blueprint.core:1.7.1]
        at org.apache.aries.blueprint.container.BlueprintEventDispatcher.callListeners(BlueprintEventDispatcher.java:189)[83:org.apache.aries.blueprint.core:1.7.1]
        at org.apache.aries.blueprint.container.BlueprintEventDispatcher.blueprintEvent(BlueprintEventDispatcher.java:140)[83:org.apache.aries.blueprint.core:1.7.1]
        at org.apache.aries.blueprint.container.BlueprintContainerImpl.doRun(BlueprintContainerImpl.java:424)[83:org.apache.aries.blueprint.core:1.7.1]
        at org.apache.aries.blueprint.container.BlueprintContainerImpl.run(BlueprintContainerImpl.java:276)[83:org.apache.aries.blueprint.core:1.7.1]
        at org.apache.aries.blueprint.container.BlueprintExtender.createContainer(BlueprintExtender.java:300)[83:org.apache.aries.blueprint.core:1.7.1]
        at org.apache.aries.blueprint.container.BlueprintExtender.createContainer(BlueprintExtender.java:269)[83:org.apache.aries.blueprint.core:1.7.1]
        at org.apache.aries.blueprint.container.BlueprintExtender.createContainer(BlueprintExtender.java:265)[83:org.apache.aries.blueprint.core:1.7.1]
        at org.apache.aries.blueprint.container.BlueprintExtender.modifiedBundle(BlueprintExtender.java:255)[83:org.apache.aries.blueprint.core:1.7.1]
        at org.apache.aries.util.tracker.hook.BundleHookBundleTracker$Tracked.customizerModified(BundleHookBundleTracker.java:500)[19:org.apache.aries.util:1.1.1]
        at org.apache.aries.util.tracker.hook.BundleHookBundleTracker$Tracked.customizerModified(BundleHookBundleTracker.java:433)[19:org.apache.aries.util:1.1.1]
        at org.apache.aries.util.tracker.hook.BundleHookBundleTracker$AbstractTracked.track(BundleHookBundleTracker.java:725)[19:org.apache.aries.util:1.1.1]
        at org.apache.aries.util.tracker.hook.BundleHookBundleTracker$Tracked.bundleChanged(BundleHookBundleTracker.java:463)[19:org.apache.aries.util:1.1.1]
        at org.apache.aries.util.tracker.hook.BundleHookBundleTracker$BundleEventHook.event(BundleHookBundleTracker.java:422)[19:org.apache.aries.util:1.1.1]
        at org.apache.felix.framework.util.SecureAction.invokeBundleEventHook(SecureAction.java:1179)[org.apache.felix.framework-5.6.1.jar:]
        at org.apache.felix.framework.EventDispatcher.createWhitelistFromHooks(EventDispatcher.java:730)[org.apache.felix.framework-5.6.1.jar:]
        at org.apache.felix.framework.EventDispatcher.fireBundleEvent(EventDispatcher.java:485)[org.apache.felix.framework-5.6.1.jar:]
        at org.apache.felix.framework.Felix.fireBundleEvent(Felix.java:4541)[org.apache.felix.framework-5.6.1.jar:]
        at org.apache.felix.framework.Felix.startBundle(Felix.java:2172)[org.apache.felix.framework-5.6.1.jar:]
        at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:998)[org.apache.felix.framework-5.6.1.jar:]
        at org.apache.felix.fileinstall.internal.DirectoryWatcher.startBundle(DirectoryWatcher.java:1253)[4:org.apache.felix.fileinstall:3.5.6]
        at org.apache.felix.fileinstall.internal.DirectoryWatcher.startBundles(DirectoryWatcher.java:1225)[4:org.apache.felix.fileinstall:3.5.6]
        at org.apache.felix.fileinstall.internal.DirectoryWatcher.doProcess(DirectoryWatcher.java:512)[4:org.apache.felix.fileinstall:3.5.6]
        at org.apache.felix.fileinstall.internal.DirectoryWatcher.process(DirectoryWatcher.java:361)[4:org.apache.felix.fileinstall:3.5.6]
        at org.apache.felix.fileinstall.internal.DirectoryWatcher.run(DirectoryWatcher.java:312)[4:org.apache.felix.fileinstall:3.5.6]
Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.elasticsearch.transport.client.PreBuiltTransportClient
        at org.apache.camel.component.elasticsearch5.ElasticsearchProducer.doStart(ElasticsearchProducer.java:221)
        at org.apache.camel.support.ServiceSupport.start(ServiceSupport.java:61)
        at org.apache.camel.util.ServiceHelper.startService(ServiceHelper.java:75)
        at org.apache.camel.impl.DeferServiceStartupListener.onCamelContextStarted(DeferServiceStartupListener.java:49)
        at org.apache.camel.impl.DefaultCamelContext.safelyStartRouteServices(DefaultCamelContext.java:3629)
        at org.apache.camel.impl.DefaultCamelContext.doStartOrResumeRoutes(DefaultCamelContext.java:3408)
        at org.apache.camel.impl.DefaultCamelContext.doStartCamel(DefaultCamelContext.java:3262)
        at org.apache.camel.impl.DefaultCamelContext.access$000(DefaultCamelContext.java:201)
        at org.apache.camel.impl.DefaultCamelContext$2.call(DefaultCamelContext.java:3046)
        at org.apache.camel.impl.DefaultCamelContext$2.call(DefaultCamelContext.java:3042)
        at org.apache.camel.impl.DefaultCamelContext.doWithDefinedClassLoader(DefaultCamelContext.java:3065)
        at org.apache.camel.impl.DefaultCamelContext.doStart(DefaultCamelContext.java:3042)
        at org.apache.camel.support.ServiceSupport.start(ServiceSupport.java:61)
        at org.apache.camel.impl.DefaultCamelContext.start(DefaultCamelContext.java:2980)
        at org.apache.camel.blueprint.BlueprintCamelContext.start(BlueprintCamelContext.java:244)
        at org.apache.camel.blueprint.BlueprintCamelContext.maybeStart(BlueprintCamelContext.java:284)
        at org.apache.camel.blueprint.BlueprintCamelContext.blueprintEvent(BlueprintCamelContext.java:177)
        at org.apache.aries.blueprint.container.BlueprintEventDispatcher$4.call(BlueprintEventDispatcher.java:201)
        at org.apache.aries.blueprint.container.BlueprintEventDispatcher$4.call(BlueprintEventDispatcher.java:199)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)[:1.8.0_121]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)[:1.8.0_121]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)[:1.8.0_121]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)[:1.8.0_121]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)[:1.8.0_121]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)[:1.8.0_121]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)[:1.8.0_121]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)[:1.8.0_121]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)[:1.8.0_121]
        at java.lang.Thread.run(Thread.java:745)[:1.8.0_121]

Any idea how to fix the issue?

Note : I have tried with feature available in 2.20 but it seems not working. here is the problem. I Basically following two bundles have been used there.

Elasticsearch-Bundle mvnrepository.com/artifact/org.apache.servicemix.bundles/org.apache.servicemix.bundles.elasticsearch/5.4.3_1

Elasticsearch-Client Bundle mvnrepository.com/artifact/org.apache.servicemix.bundles/org.apache.servicemix.bundles.elasticsearch-client/5.4.3_1

In order to work with elasticsearch 5 you need TransportClient as mentioned in example which is available in Elasticsearch bundle
To get the TransportClient for first step you need to use PreBuiltTransportClient which is available in Elasticsearch-Client bundle.
Now the problem is that Elasticsearch-Client which is expected to return TransportClient does not have it and also not importing it and gives class not found exception " Caused by: java.lang.ClassNotFoundException: org.elasticsearch.client.transport.TransportClient not found by org.apache.servicemix.bundles.elasticsearch-client [127]"

Not everything works/runs in OSGi. Can you try to dig deeper to see if you can get more than just that stacktrace? Maybe check the source code of elastichseach, and turn on debug logging etc and see what goes on. — Claus Ibsen, Jul 19 '17 at 07:29
This looks a well known issue and has been already reported but not fixed yet. [Camel-Community](http://camel.465427.n5.nabble.com/problem-when-using-elasticsearch5-component-with-camel-2-19-0-td5799247.html), [Servicemix-Community](http://servicemix.396122.n5.nabble.com/Problem-when-using-elasticsearch5-component-with-camel-2-19-0-td5724431.html) — anna, Jul 19 '17 at 11:19

Oscerd · Answer 1 · 2017-08-04T07:51:21.183

1

This should be fixed with bundle version 5.3.2_1 from Servicemix and in Apache Camel 2.20.0.

In Camel 2.19.x branch we have the following feature for Karaf

https://github.com/apache/camel/blob/camel-2.19.x/platforms/karaf/features/src/main/resources/features.xml#L508-L522

While in 2.20.x branch we have a bundle for Elasticsearch client too:

https://github.com/apache/camel/blob/master/platforms/karaf/features/src/main/resources/features.xml#L566-L580

You can try with Camel 2.20.0-SNAPSHOT version.

UPDATE:

There is a JIRA issue about this: https://issues.apache.org/jira/browse/SM-3454 and we're going to fix it

edited Aug 04 '17 at 07:51

answered Jul 24 '17 at 07:24

Oscerd

1,616
11
14

How can I try 2.20.0-SNAPSHOT as it is not available yet on [maven repo](https://repo1.maven.org/maven2/org/apache/camel/karaf/apache-camel/)? – anna Jul 31 '17 at 13:18
You can build the source code on your machine or use the following repository http://repository.apache.org/snapshots . This repository isn't always updated. – Oscerd Aug 01 '17 at 06:17
I have updated question regarding your suggestion to use 2.20.x stuff. – anna Aug 01 '17 at 15:05
I'll get back to you soon, maybe we can fix this. – Oscerd Aug 02 '17 at 05:52
We opened a JIRA about this https://issues.apache.org/jira/browse/SM-3454 answer edited – Oscerd Aug 04 '17 at 07:50
Can you try with camel 2.21.0-SNAPSHOT? We changed the Elasticsearch bundle and we are using a new one. – Oscerd Nov 13 '17 at 13:25

camel-elasticsearch5 component is not working

1 Answers1