1

I am trying to connect with two elasticsearch clusters using scala code and query elasticsearch from them. using following libraries and code in build.sbt of scala project:

libraryDependencies +="org.elasticsearch" % "elasticsearch" % "7.2.0"
libraryDependencies += "org.elasticsearch.client" % "elasticsearch-rest-high-level-client" % "7.2.0"

AND

val elastic4sVersion = "6.2.8"
libraryDependencies ++= Seq(
  "com.sksamuel.elastic4s" %% "elastic4s-core" % elastic4sVersion,
  // for the http client
  "com.sksamuel.elastic4s" %% "elastic4s-http" % elastic4sVersion,
)

These has a common client library, which gets missing when building. I can see either 6.x or 7.x but not both. I tried the shading approach

assemblyShadeRules in assembly := Seq(
  ShadeRule.rename("org.elasticsearch.client.**" -> "my_conf.@1")
    .inLibrary("org.elasticsearch.client" % "elasticsearch-rest-high-level-client" % "7.2.0")
    .inAll
)

assemblyShadeRules in assembly := Seq(
  ShadeRule.rename("org.elasticsearch.client.**" -> "my_conf_1.@1")
    .inLibrary("org.elasticsearch.client" % "elasticsearch-rest-high-level-client" % "6.2.2")
    .inAll
)
assemblyShadeRules in assembly := Seq(
  ShadeRule.rename("org.elasticsearch.elasticsearch.**" -> "my_configuration.@1")
    .inLibrary("org.elasticsearch" % "elasticsearch" % "7.2.0")
      .inAll
)

but I can not can not get the shaded versions available and get error when trying to import them in projet references.

Ashutosh C
  • 21
  • 5

1 Answers1

1

Well, on JVM you can have only one version of the same .class in a classpath, so every build tool will respect that.

sbt will make sure there will be only one version of the library available in the project (unless you set it explicitly, I would assume it picks the highest version number of all conflicting versions) so if you need to have a library that is used by both dependencies I would look for versions that use the same version of the dependency. (Or explicitly override the version and use sbt-mima with sbt-missinglink to check that this change didn't broke anything).

From what I see the simplest way would be to use the table on projects page (together with maven) to pick up the right elastic4s version for the elasticsearch version you want to use.

Mateusz Kubuszok
  • 24,995
  • 4
  • 42
  • 64
  • They have changes the configuration for clients connection from es6 to es7. I am unable to find one library that can server for both the elastic4s 6.2 and the Rest high level client connection to connect using elasticsearch7 in the same project. – Ashutosh C Jul 10 '20 at 11:36
  • 1
    You CANNOT use TWO versions of the same library in the same project. Period. If you need to use ES6, then table shows that 6.7.7 is the last version supporting ES6. And there is plenty of "high level" integrations for it with that version number https://search.maven.org/search?q=g:com.sksamuel.elastic4s%20AND%20v:6.7.7 - if you need 7.8.0 (or 7.2.0 and what not) then there are integrations as well: https://search.maven.org/search?q=g:com.sksamuel.elastic4s%20AND%20v:7.8.0 – Mateusz Kubuszok Jul 10 '20 at 11:55