Spark submit in IntelliJ of project jar yields Class not found error

Question

I'm doing a basic program of implementing Drools and the program runs on an application configuration but when I try to run the JAR, I face an error.

The error I get on the terminal:

    `Suhita-MacBookPro:Drool-CreditScore-Sample sgoswami$ spark-submit --class main.scala.suhita.Sample --master local[*] target/DroolsMaven-1.0-SNAPSHOT.jar 
java.lang.ClassNotFoundException: main.scala.suhita.Sample
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at org.apache.spark.util.Utils$.classForName(Utils.scala:230)
    at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:712)
    at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
    at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119)
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)`


Drools-Project
  >src
   >main
    >scala
     >suhita
      - Sample
      - Applicant
   >META-INF
    -kmodule.xml
    -manifest.MF
   >resources.rules
    -rules

A class is not found in the classpath. Who knows what is going on because you did not provide a [mcve]. Please read [ask]. — , Mar 08 '18 at 20:29

score 0 · Answer 1 · answered Mar 08 '18 at 22:14

This happens sometime when your classes are not loaded property. I have seen a few times recently. So there are two ways to fix this issue:

Refresh classes that may include sbt clean compile
or you can try reloading idea classes from top menu. It's very vague to say but sometimes restarting intellij also works as it reloads all the classes again.

I am sure one the method will work. Let me know if it persists.

score 0 · Answer 2 · answered Mar 09 '18 at 04:20

Looking at the tree structure of your project

Drools-Project
  >src
   >main
    >scala
     >suhita
      - Sample
      - Applicant

You don't need to provide class name from main as main.scala.suhita.Sample

Simply use suhita.Simple for class name as

spark-submit --class suhita.Sample --master local[*] target/DroolsMaven-1.0-SNAPSHOT.jar

and it should work

Spark submit in IntelliJ of project jar yields Class not found error

2 Answers2