1

I tried installing Spark on windows 10. I followed the steps in this order:

  1. Installed Java (outside Program Files folder in C drive)
  2. Validated the version of spark downloaded from Apache(spark-3.2.0-bin-hadoop3.2.tgz)
  3. unzip the spark in the folder outside Program files(Installed in C drive)
  4. Downloaded winutils.exe (from GIT which is in the folder Hadoop-3.2.0/bin) and put that in c:/hadoop/bin folder
  5. Set the environment variables for JAVA_HOME (path of java), SPARK_HOME (path of the spark installation), HADOOP_HOME (path of winutils)
  6. Included the PATH variable with %JAVA_HOME%/bin and similarly the other 2.

When I tried running Spark -version, it gives an error Spark is not recognized as the internal or external command. When I run spark-shell, it gives the error SparkContext: Error initializing SparkContext / Utils: Uncaught exception in thread main / ERROR Main: Failed to initialize Spark session.

Could you please let me know if I missed any steps for successful execution? Any suggestions on how to resolve these errors while running spark?

Oli
  • 9,766
  • 5
  • 25
  • 46
Anu
  • 11
  • 1
  • Have you checked that your version of Java is compatible? – tjheslin1 Nov 04 '21 at 07:27
  • Thanks! I installed Java 8 version earlier but I updated the java Version to 17 and now I do not see the above error and the Spark shell starts. But with these errors. "module java. base does not export sun.nio.ch to the unnamed module " and import spark.implicits._ - Value spark not found. – Anu Nov 05 '21 at 13:06

0 Answers0