4

I am working with flink and kafka, i got this error.

java.lang.NoSuchMethodError: scala.Predef$.ArrowAssoc(Ljava/lang/Object;)Ljava/lang/Object;
    at kafka.consumer.FetchRequestAndResponseMetrics.<init>(FetchRequestAndResponseStats.scala:32)
    at kafka.consumer.FetchRequestAndResponseStats.<init>(FetchRequestAndResponseStats.scala:46)
    at kafka.consumer.FetchRequestAndResponseStatsRegistry$$anonfun$2.apply(FetchRequestAndResponseStats.scala:59)
    at kafka.consumer.FetchRequestAndResponseStatsRegistry$$anonfun$2.apply(FetchRequestAndResponseStats.scala:59)
    at kafka.utils.Pool.getAndMaybePut(Pool.scala:61)
    at kafka.consumer.FetchRequestAndResponseStatsRegistry$.getFetchRequestAndResponseStats(FetchRequestAndResponseStats.scala:63)
    at kafka.consumer.SimpleConsumer.<init>(SimpleConsumer.scala:39)
    at kafka.javaapi.consumer.SimpleConsumer.<init>(SimpleConsumer.scala:34)
    at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer.getPartitionsForTopic(FlinkKafkaConsumer.java:695)
    at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer.<init>(FlinkKafkaConsumer.java:281)
    at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer082.<init>(FlinkKafkaConsumer082.java:49)
    at com.inndata.flinkkafka.ReadFromKafka.main(ReadFromKafka.java:19)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:505)
    at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:403)
    at org.apache.flink.client.program.Client.runBlocking(Client.java:248)
    at org.apache.flink.client.CliFrontend.executeProgramBlocking(CliFrontend.java:866)
    at org.apache.flink.client.CliFrontend.run(CliFrontend.java:333)
    at org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:1192)
    at org.apache.flink.client.CliFrontend.main(CliFrontend.java:1243)

i am using flink - 1.0.3 version and Kafka- kafka_2.11-0.8.2.1 and scala - 2.11.5, these are the jars i am using in my buildpath :

asm-4.0.jar,commons-codec-1.6.jar,commons-exec-1.1.jar,commons-fileupload-1.2.1.jar,commons-io-2.4.jar,commons-lang-2.5.jar,commons-logging-1.1.3.jar,commons-logging-api-1.1.jar,curator-test-3.2.0.jar,disruptor-2.10.1.jar,flink-annotations-1.0.3.jar,flink-clients_2.10-1.0.3.jar,flink-clients_2.10-1.0.3-tests.jar,flink-core-1.0.3.jar,flink-dist_2.10-1.0.3.jar,flink-java-1.0.3.jar,flink-optimizer_2.10-1.0.3.jar,flink-python_2.10-1.0.3.jar,flink-runtime_2.10-1.0.3.jar,flink-test-utils_2.10-1.0.3.jar,guava-11.0.2.jar,hadoop-client-2.6.0.jar,hadoop-mapreduce-client-core-2.6.0.jar,hamcrest-all-1.3.jar,jetty-6.1.26.jar,jetty-util-6.1.26.jar,json-simple-1.1.1.jar,junit-4.11.jar,log4j-1.2.17.jar,logback-classic-1.0.13.jar,logback-core-1.0.13.jar,slf4j-api-1.7.7.jar,slf4j-log4j12-1.7.7.jar,zkclient-0.3.jar,zookeeper-3.4.6.jar,flink-connector-kafka-0.10.2.jar,flink-streaming-java-0.10.2.jar,kafka-clients-0.8.2.1.jar,kafka_2.11-0.8.2.1-test.jar,kafka_2.11-0.8.2.1-sources.jar,kafka_2.11-0.8.2.1-scaladoc.jar,kafka_2.11-0.8.2.1-javadoc.jar,kafka_2.11-0.8.2.1.jar,scala-xml_2.11-1.0.2.jar,scala-parser-combinators_2.11-1.0.2.jar,scala-library-2.11.5.jar,metrics-core-2.2.0.jar

I tried searching for this issue, but i can't find relevent jar.Please help me to fix this.

kosa
  • 65,990
  • 13
  • 130
  • 167
9966
  • 51
  • 1
  • 1
  • 5
  • 3
    Most of the NoSuchMethodErrors are due to the fact that at development/compile time you used one of version of jar and runtime you are using another version of jar (which doesn't contain the method with exact syntax you are looking for). Check jar versions between compile time & run time. – kosa Jun 30 '16 at 15:18
  • http://stackoverflow.com/questions/25089852/what-is-the-reason-for-java-lang-nosuchmethoderror-scala-predef-arrowassoc-upo – meucaa Jun 30 '16 at 15:19
  • There are `_2.10` and `_2.11` scala libs in the classpath, those are not compatible – knutwalker Jun 30 '16 at 15:21
  • Possible duplicate of [java.lang.NoSuchMethodError in Flink](https://stackoverflow.com/questions/46920543/java-lang-nosuchmethoderror-in-flink) – Yaroslav Oct 25 '17 at 12:45
  • @kosa How do I check that? – pavel_orekhov Jan 28 '19 at 09:39

3 Answers3

4

There are a few jars that might be the cause of the issue. scala-library-2.11.5.jar suggests you are using Scala 2.11, this seems consistent with kafka_2.11-0.8.2.1.jar.

But, also in your CP, I see flink-clients_2.10-1.0.3.jar and flink-runtime_2.10-1.0.3.jar among others. These are scala libraries that were compiled against Scala 2.10. That's the problem.

Binaries in Scala are not compatible across major versions. You need to find the 2.11 version of these jars if they exist or compile them to 2.11 yourself.

pedrofurla
  • 12,763
  • 1
  • 38
  • 49
0

  1. Navigate to Scala folder under your project folder, e.g. src -> test -> scala

  2. Right click on Scala directory and choose mark directory as Test Sources Root

  3. Now mark mark src directory as Sources Root

William Baker Morrison
  • 1,642
  • 4
  • 21
  • 33
Rishabh Jain
  • 1
0

Facing same issue in my codebase as well when trying to trigger a simulation using Runner class, but when triggering using the below maven command this looks fine,

mvn gatling:test -Dgatling.simulationClass=simulations.simulationClassName

this is just because of version mismatch

things to check before running it using Runner class(Worked for me)

  1. Check if java is correctly configured and reflect the version (1.8.0_251 in my case)
  2. Check if maven is correctly set (3.6.3 pointing to the same java version as above)
  3. If you're using IntelliJ, make sure you've added scala SDK in global libraries(2.12.8 in my case)
  4. Once all configured rebuild your project
  5. Trigger your simulation using Runner now, should work.
Rishabh Jain
  • 1