-1

My spark-submit command is :

spark-submit --class com.sundogsoftware.spark.WordCountBetterDataset --master yarn --deploy-mode cluster SparkCourse.jar

And for defining the sparkSession, i use this :

val spark = SparkSession
  .builder
  .master("spark://youness:7077")
  .appName("WordCount")
  .getOrCreate()

but at the end, my job fails with return code 13.

YounessGI
  • 101
  • 3

1 Answers1

0

You need to let the master unset in the code. It is preferable to set it later when you issue spark-submit (spark-submit --master yarn-client ...) and you are already doing that above. Just remove .master("spark://youness:7077") from your code.

1218985
  • 7,531
  • 2
  • 25
  • 31