0

I am running a Kafka server. (when i use the command bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning it gives me all ma data in my topic).

When I want to test the example JavaDirectKafkaWordCount in spark in order to understand how it works i get the following error:

$ ./run-example streaming.JavaDirectKafkaWordCount localhost:2181 test Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties 16/08/17 11:19:33 INFO StreamingExamples: Setting log level to [WARN] for streaming example. To override add a custom log4j.properties to the classpath. 16/08/17 11:19:33 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 16/08/17 11:19:33 WARN Utils: Your hostname, localhost.localdomain resolves to a loopback address: 127.0.0.1; using 10.66.212.132 instead (on interface enp5s0) 16/08/17 11:19:33 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address Exception in thread "main" org.apache.spark.SparkException: java.io.EOFException: Received -1 when reading from channel, socket has likely been closed. at org.apache.spark.streaming.kafka.KafkaCluster$$anonfun$checkErrors$1.apply(KafkaCluster.scala:366) at org.apache.spark.streaming.kafka.KafkaCluster$$anonfun$checkErrors$1.apply(KafkaCluster.scala:366) at scala.util.Either.fold(Either.scala:97) at org.apache.spark.streaming.kafka.KafkaCluster$.checkErrors(KafkaCluster.scala:365) at org.apache.spark.streaming.kafka.KafkaUtils$.getFromOffsets(KafkaUtils.scala:222) at org.apache.spark.streaming.kafka.KafkaUtils$.createDirectStream(KafkaUtils.scala:484) at org.apache.spark.streaming.kafka.KafkaUtils$.createDirectStream(KafkaUtils.scala:607) at org.apache.spark.streaming.kafka.KafkaUtils.createDirectStream(KafkaUtils.scala) at org.apache.spark.examples.streaming.JavaDirectKafkaWordCount.main(JavaDirectKafkaWordCount.java:71) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731) at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181) at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

I would like to know what the error means and how i could possible solve it.

Thank you very much for your attention and your help.

Joe

Yassir S
  • 1,032
  • 3
  • 21
  • 44
  • is your kafka secure ? What version of kakfa you are using?? – m-bhole Aug 17 '16 at 13:51
  • My kafka is not secure and i am using the version kafka_2.11-0.10.0.1 and 1.6.1 for apache spark – Yassir S Aug 17 '16 at 14:03
  • follow the links : a] http://stackoverflow.com/questions/30932568/spark-streaming-kafka-integration-direct-approach-eofexception and b] http://apache-spark-developers-list.1001551.n3.nabble.com/Spark-Kafka-Connector-issue-td14381.html – m-bhole Aug 17 '16 at 15:42

0 Answers0