Reindex elasticsearch 2.3.3 when using Java NodeClient

Question

With the deprecation of SearchTye.SCAN and the newly Reindex API we want to migrate our elasticsearch cluster and clients from 2.1.1 to 2.3.3.

We use java and the appropiate libraries to access elasticsearch. To access the cluster we use the TransportClient, for embedded Unittests we use the NodeClient.

Unfortunatly the Reindex API is provided as plugin, which the NodeClient seems to be unable to deal with.

So the question is how to use the NodeClient with the Reindex-Plugin?

I already tried exposing the protected NodeClient constructor to pass the ReindexPlugin class as argument without success.

Using the NodeClient to start an embedded ElasticSearch and using the TransportClient with the added ReindexPlugin didn't worked either. All I get here is an Exception: ActionNotFoundTransportException[No handler for action [indices:data/write/reindex]]

Dependencies of interest:

org.elasticsearch:elasticsearch:2.3.3
org.elasticsearch.module:reindex:2.3.3
org.apache.lucene:lucene-expressions:5.5.1
org.codehaus.groovy:groovy:2.4.6

Starting the NodeClient:

Settings.Builder settings = Settings.settingsBuilder();
settings.put("path.data", "/some/path/data");
settings.put("path.home", "/some/path/home");
//settings.put("plugin.types", ReindexPlugin.class.getName()); > No effect
settings.put("http.port", 9299);
settings.put("transport.tcp.port", 9399);

node = NodeBuilder.nodeBuilder()
    .clusterName("testcluster")
    .settings(settings)
    .local(true)
    .node();

// also tested with local(false), then no transport port is available, resulting in NoNodeAvailableException

Using TransportClient to access the Node:

Settings settings = Settings.settingsBuilder()
    .put("cluster.name", "testcluster")
    .put("discovery.zen.ping.multicast.enabled", false)
    .build();
InetSocketTransportAddress[] addresses = new InetSocketTransportAddress[]
    {new InetSocketTransportAddress(new InetSocketAddress("localhost", 9399))};
client = TransportClient.builder()
    .settings(settings)
    .addPlugin(ReindexPlugin.class)
    .build()
    .addTransportAddresses(addresses);

Main part of triggering reindex:

ReindexRequestBuilder builder = ReindexAction.INSTANCE.newRequestBuilder(getClient())
    .source(indexFrom)
    .destination(indexTo)
    .refresh(true);

score 0 · Accepted Answer · answered Jun 14 '16 at 13:16

I was able to solve this, by combining both approaches described above.

So creating the NodeClient involves overriding the Node:

class ExposedNode extends Node {
    public ExposedNode(Environment tmpEnv, Version version, Collection<Class<? extends Plugin>> classpathPlugins) {
        super(tmpEnv, version, classpathPlugins);
    }
}

And using it when starting the NodeClient:

Settings.Builder settings = Settings.settingsBuilder();
settings.put("path.data", "/some/path/data");
settings.put("path.home", "/some/path/home");
settings.put("http.port", 9299);
settings.put("transport.tcp.port", 9399);

// Construct Node without NodeBuilder
List<Class<? extends Plugin>> classpathPlugins = ImmutableList.of(ReindexPlugin.class);
settings.put("node.local", false);
settings.put("cluster.name", "testcluster");
Settings preparedSettings = settings.build();
node = new ExposedNode(InternalSettingsPreparer.prepareEnvironment(preparedSettings, null), Version.CURRENT, classpathPlugins);
node.start();

After that you can use the TransportClient which adds the ReindexPlugin, as described in the question.

Nevertheless this is a dirty hack, which may break in a future release, and shows how poorly Elasticsearch supports plugin development imo.

Reindex elasticsearch 2.3.3 when using Java NodeClient

1 Answers1