Is there an info on compatibility beween java elasticsearch libs and es distros?

Question

I looked for this but had no success. Everything I found was this, which doesn't contain any information about compatibility with version of java es libs.

Answer 1

It depends on what you mean by "client".

Historical Java API

Otherwise named "Java API", or "Transport client". This client has been around since the earliest versions. You can read on the documentation :

The TransportClient is deprecated in favour of the Java High Level REST Client and will be removed in Elasticsearch 8.0. The migration guide describes all the steps needed to migrate.

Following up on the migration guide, you end up on this page (emphasis mine) :

Motivations around a new Java client

The existing TransportClient has been part of Elasticsearch since its very first commit. It is a special client as it uses the transport protocol to communicate with Elasticsearch, which causes compatibility problems if the client is not on the same version as the Elasticsearch instances it talks to.

Conclusion : the Java API client needs to follow the server's version.

Java Low Level Client

Introduced in 2016 with ES 5.0, the low level client really is just a glorified raw HTTP client with a few tricks to better integrate to ES.

The first release only included what we call a low-level client, which has the following features:

• compatibility with any Elasticsearch version
• load balancing across all available nodes
• failover in case of node failures and upon specific response codes
• failed connection penalization
• persistent connections
• trace logging of requests and responses
• optional automatic discovery of cluster nodes (also known as sniffing)

As it only really handles HTTP-level stuff, the low level client is compatible accross all verions. On the other hand, it does not help you write/format JSON SearchDSL queries or admin queries, for that, you need... the high level client.

It is called the low-level client because it does little to help the Java users to build requests or to parse responses. It handles path and query-string construction for requests, but it treats JSON request and response bodies as opaque byte arrays which have to be handled by the user.

The next step is releasing a high level client that accepts proper request objects, takes care of their marshalling, and returns parsed response objects.

Java High Level Client

The High Level client has a compatibility page.

Long story short : you should use a version that is at least equal to the cluster version. Lagging a minor version is usually fine, but it is not expected. Lagging a major version may not work, as major version may introduce API changes.

The High Level Client is guaranteed to be able to communicate with any Elasticsearch node running on the same major version and greater or equal minor version. It doesn’t need to be in the same minor version as the Elasticsearch nodes it communicates with, as it is forward compatible meaning that it supports communicating with later versions of Elasticsearch than the one it was developed for.

The 6.0 client is able to communicate with any 6.x Elasticsearch node, while the 6.1 client is for sure able to communicate with 6.1, 6.2 and any later 6.x version, but there may be incompatibility issues when communicating with a previous Elasticsearch node version, for instance between 6.1 and 6.0, in case the 6.1 client supports new request body fields for some APIs that are not known by the 6.0 node(s).

It is recommended to upgrade the High Level Client when upgrading the Elasticsearch cluster to a new major version, as REST API breaking changes may cause unexpected results depending on the node that is hit by the request, and newly added APIs will only be supported by the newer version of the client.