0

I've been trying to create an integration test using the embeddedKafka, but I'm getting problem of missing dependency when trying to run it, this is the error:

Unable to load class org.springframework.kafka.test.EmbeddedKafkaBroker due to missing dependency org/I0Itec/zkclient/serialize/ZkSerializer

I saw some stuff saying that this is related to my dependencies, so here is my dependencies:

springBootVersion = '2.3.5.RELEASE'

compile("org.springframework.boot:spring-boot-starter-web:${springBootVersion}")
compile("org.springframework.kafka:spring-kafka:${springBootVersion}")
testCompile("org.springframework.boot:spring-boot-starter-test:${springBootVersion}",
            'org.spockframework:spock-core:1.2-groovy-2.4',
            'org.spockframework:spock-spring:1.2-groovy-2.4',
            'com.microsoft.azure:spring-data-cosmosdb:2.3.0',
            'com.nimbusds:oauth2-oidc-sdk:5.64.4',
    )
testCompile("org.springframework.kafka:spring-kafka-test:${springBootVersion}")

So, my question is, am I missing something?

EDIT

After changed the versions as indicated, I got a different error:

Error creating bean with name 'embeddedKafka': Invocation of init method failed; nested exception is java.lang.NoClassDefFoundError: scala/math/Ordering$$anon$7

I've added the scala dependencies, but still having the same issue:

testImplementation("org.scala-lang:scala-library:2.12.11")
testImplementation("org.scala-lang:scala-reflect:2.12.11")
Pedro Henrique
  • 609
  • 2
  • 9
  • 26

2 Answers2

0

You somehow have a mismatched kafka Vs. kafka-clients jars on the classpath; they all must be the same version.

You generally should not specify a version on boot's dependencies and use its dependency management instead.

You are pulling in spring-kafka 2.3.5 whereas spring-boot 2.3.5 requires spring-kafka 2.5.7.

Spring-kafka 2.5.x uses the kafka-clients 2.5.1.

See here for how to override versions of kafka jars when using a different version to the version that Boot prescribes.

Gary Russell
  • 166,535
  • 14
  • 146
  • 179
  • Hm, so instead of spring-kafka 2.3.5 I have to use the 2.5.7? I'll try it – Pedro Henrique Apr 09 '21 at 16:41
  • 1
    Yes, but it's better to let boot manage its depencies https://docs.spring.io/spring-boot/docs/2.3.9.RELEASE/reference/html/using-spring-boot.html#using-boot-dependency-management – Gary Russell Apr 09 '21 at 16:42
  • `testCompile("org.springframework.kafka:spring-kafka-test:${springBootVersion}")` - that's wrong. The `spring-kafka-test` is not a part of Spring Boot project. It is released with its own version. You definitely need to let Spring Boot to manage its dependency versions. – Artem Bilan Apr 09 '21 at 16:43
  • I have changed the kafka dependencies, and now I got a different error: `Error creating bean with name 'embeddedKafka': Invocation of init method failed; nested exception is java.lang.NoClassDefFoundError: scala/math/Ordering$$anon$7` I edited the question.. – Pedro Henrique Apr 09 '21 at 16:46
  • This is another mismatch - something else (probably Jackson) is pulling the wrong kafka jar. See https://docs.spring.io/spring-kafka/docs/2.5.12.RELEASE/reference/html/#jacksonscala-incompatibility The scala folks are not very good at backwards compatibility in patch releases. – Gary Russell Apr 09 '21 at 16:56
  • Sure, I added the scala libraries and now i'm doing a refresh on dependencies, will come back with the result in some minutes, thanks @GaryRussell – Pedro Henrique Apr 09 '21 at 17:02
  • Hey @GaryRussell, I added the scala dependencies, refreshed the gradle dependencies and cleanup the intellij cache but still getting the same error of missing scala class, do you have any other recommendation of what I could do? – Pedro Henrique Apr 09 '21 at 17:14
  • Use `./gradlew :dependencies` on the command line to figure out what's happening. – Gary Russell Apr 09 '21 at 17:18
  • Worked after change from testImplementation to testCompile on gradle! – Pedro Henrique Apr 09 '21 at 17:21
0

the Kafka Client libraries for a time were inlining a particular version of the Scala library. This caused problems for those of us wanting to use the kafka client library with a slightly different version of Scala than that inline version.

In this cases the version of Scala they were using inline is Scala 2.12.10

They removed this dependency in later versions and this was backported as fixes (the earliest being 2.8.0) https://archive.apache.org/dist/kafka/2.8.0/RELEASE_NOTES.html)

Andrew Norman
  • 843
  • 9
  • 22