Spark not working with error "java.lang.UnsatisfiedLinkError: no snappyjava in java.library.path"

Asked Jan 18 '15 at 10:56

Active Jan 22 '15 at 18:06

Viewed 1,809 times

I am using the CDH4 build of Spark 1.2.0 to submit a jar of Scala code built using sbt assembly.

Both spark shell and spark submit are failing with below error.

java.lang.UnsatisfiedLinkError: no snappyjava in java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1738)
        at java.lang.Runtime.loadLibrary0(Runtime.java:823)
        at java.lang.System.loadLibrary(System.java:1028)

Though I have included snappy as a dependency in build.sbt as below, and it gets downloaded too.

libraryDependencies ++= Seq(
  // Spark dependency
  "org.apache.spark" % "spark-core_2.10" % "1.2.0" % "provided",
  "org.xerial.snappy" % "snappy-java" % "1.0.5.4"
)

Please suggest, what is the issue and appropriate resolution.

Here is the spark-submit command

./spark-submit --class org.cmls.SparkExamples.Spark_First /home/varun/SPARK/spark-1.2.0/assembly/target/scala-2.11/my-project-assembly.jar

edited Jan 22 '15 at 18:06

asked Jan 18 '15 at 10:56

Varun

Is snappy-java : - on your executors ? (do you use `--jars` with `spark-submit` ?) - on your classpath ? (how do you build your `SparkConfig`) – Francois G Jan 19 '15 at 11:09
Please include your `spark-submit` command in the question. – Daniel Darabos Jan 19 '15 at 19:22
I am using a downloaded, pre-built package , spark-1.2.0-bin-cdh4. – Varun Jan 22 '15 at 18:11

Spark not working with error "java.lang.UnsatisfiedLinkError: no snappyjava in java.library.path"

0 Answers0