3

I am using spark-submit to execute a jar file. Spark is located in my "C" drive and my eclipse workspace is in "D" drive. Tough I am giving an absolute path for the jar file I get the error saying "The filename, directory name, or volume label syntax is incorrect."

Below is the command i used to run spark-submit:

C:\spark-2.1.0-bin-hadoop2.7\bin>spark-submit --class "Main" --master local[2] "
D:\JAVA NEW ENV\Line_Count_Spark\target\Line_Count_Spark-0.0.1-SNAPSHOT.jar"

What is exactly wrong with this and is there any workaround for the same?

Yeshwanth
  • 70
  • 8
  • 1
    `C:\spark-2.1.0-bin-hadoop2.7\bin>spark-submit --class "Main" --master local[2] jar/location`. No quotes in path – mrsrinivas Feb 10 '17 at 11:25
  • Actually I have spaces in my workspace folder name. I edited the earlier mentioned path above – Yeshwanth Feb 10 '17 at 11:29
  • `C:\spark-2.1.0-bin-hadoop2.7\bin>spark-submit --class "Main" --master local[2] D:\JAVA^ NEW^ ENV\Line_Count_Spark\target\Line_Count_Spark-0.0.1-SNAPSHOT.jar`. escaping the space with a `caret symbol`. – mrsrinivas Feb 10 '17 at 11:31
  • Still it is throwing the error 'C:\spark-2.1.0-bin-hadoop2.7\bin\spark-submit2.cmd" --class "Main' is not recog nized as an internal or external command, operable program or batch file. – Yeshwanth Feb 10 '17 at 11:38
  • specify class name along with package for example `com.test.Main` where `com.test` is my package and `Main` is my class where I have `main()` – Prasad Khode Feb 10 '17 at 11:54
  • My package is default package. I tried giving it but it shows the same error `C:\spark-2.1.0-bin-hadoop2.7\bin>spark-submit --class "default.Main" --master local[2] C:\test\Line_Count_Spark-0.0.1-SNAPSHOT.jar 'C:\spark-2.1.0-bin-hadoop2.7\bin\spark-submit2.cmd" --class "default.Main' is not recognized as an internal or external command, operable program or batch file.` – Yeshwanth Feb 10 '17 at 12:03
  • It worked when I removed quotes for class name. I tried it without giving package name. Thanks for quick reply – Yeshwanth Feb 10 '17 at 12:07

1 Answers1

3

try "spark-submit2"

spark-submit2 --class "Main" --master local[2] "D:\JAVA NEW ENV\Line_Count_Spark\target\Line_Count_Spark-0.0.1-SNAPSHOT.jar"
birkan
  • 41
  • 5
  • we faced the same problem when we pass multiple jars in double quotes with comma separated, spark-submit.cmd throwing an the filename, directory name, or volume label syntax is incorrect. even though spark-submit.cmd internally calls spark-submit2.cmd seems some issue. the moment switching to spark-submit2 every thing fine. – Ankireddy Polu Jun 28 '17 at 06:39