2

I'm writing a simple scala project for dl4j. I need to switch between cuda (for training) and native for production. I seem to have a problem using native in an assembled jar. I get the below error:

Exception in thread "main" java.lang.ExceptionInInitializerError
    at org.datavec.api.util.ndarray.RecordConverter.toMinibatchArray(RecordConverter.java:197)
    at org.deeplearning4j.datasets.datavec.RecordReaderMultiDataSetIterator.next(RecordReaderMultiDataSetIterator.java:159)
    at org.deeplearning4j.datasets.datavec.RecordReaderDataSetIterator.next(RecordReaderDataSetIterator.java:364)
    at org.deeplearning4j.datasets.datavec.RecordReaderDataSetIterator.next(RecordReaderDataSetIterator.java:439)
    at simplecuda.main$.delayedEndpoint$simplecuda$main$1(main.scala:37)
    at simplecuda.main$delayedInit$body.apply(main.scala:27)
    at scala.Function0$class.apply$mcV$sp(Function0.scala:34)
    at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
    at scala.App$$anonfun$main$1.apply(App.scala:76)
    at scala.App$$anonfun$main$1.apply(App.scala:76)
    at scala.collection.immutable.List.foreach(List.scala:381)
    at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:35)
    at scala.App$class.main(App.scala:76)
    at simplecuda.main$.main(main.scala:27)
    at simplecuda.main.main(main.scala)
Caused by: java.lang.RuntimeException: org.nd4j.linalg.factory.Nd4jBackend$NoAvailableBackendException: Please ensure that you have an nd4j backend on your classpath. Please see: http://nd4j.org/getstarted.html
    at org.nd4j.linalg.factory.Nd4j.initContext(Nd4j.java:5449)
    at org.nd4j.linalg.factory.Nd4j.<clinit>(Nd4j.java:213)
    ... 15 more
Caused by: org.nd4j.linalg.factory.Nd4jBackend$NoAvailableBackendException: Please ensure that you have an nd4j backend on your classpath. Please see: http://nd4j.org/getstarted.html
    at org.nd4j.linalg.factory.Nd4jBackend.load(Nd4jBackend.java:213)
    at org.nd4j.linalg.factory.Nd4j.initContext(Nd4j.java:5446)
    ... 16 more

My build file is:

name := "simplecuda"
version := "1.0"
scalaVersion := "2.11.8"


// looks like you need to remove ~/.ivy2/cache and ~/.javacpp/cache whenever you switch between platforms 
classpathTypes += "maven-plugin"
libraryDependencies ++= Seq(
    "org.scalactic" %% "scalactic" % "3.0.5",
    "org.scalatest" %% "scalatest" % "3.0.5" % "test",

    // "org.nd4j" % "nd4j-cuda-9.2-platform" % "1.0.0-beta2",   
    // "org.deeplearning4j" % "deeplearning4j-cuda-9.2" % "1.0.0-beta2"

    "org.nd4j" % "nd4j-native-platform" % "1.0.0-beta2",
    "org.deeplearning4j" % "deeplearning4j-core" % "1.0.0-beta2"
)

assemblyMergeStrategy in assembly := {
    case PathList("META-INF", xs @ _*) => MergeStrategy.discard
    case x => MergeStrategy.first
}   

When I visit http://nd4j.org/getstarted.html to learn about the noAvailableBackendException I see that build.sbt should have the below line:

classpathTypes += "maven-plugin"

I've included this in the above build.sbt and without any luck. After looking at the gradle instructions I tried adding the "org.bytedeco.javacpp-presets" % "openblas" % "0.2.20-1.3" classifier "linux-x86_64" dependency and this also did not help.

I've tried removing ~/.javacpp/cache and ~/.ivy2/cache multiple times without any luck. The repo with this example is at https://github.com/tomlue/dl4j_scala_troubleshoot

Thomas Luechtefeld
  • 1,316
  • 15
  • 23

0 Answers0