-1

I am using Jupiter Notebook in Anaconda.

I have managed to install pyspark and findpyspark.

I am now using the code: SPARK_HOME = C:\spark\spark-2.3.2-bin-hadoop2.7 to set sparks home.

However, I get the error: 

File "<ipython-input-27-2126598a4c18>", line 1
    SPARK_HOME  = C:\spark\spark-2.3.2-bin-hadoop2.7
                   ^
SyntaxError: invalid syntax

How can I resolve this?

Micah Ondiwa
  • 59
  • 9

1 Answers1

1

You just need to specify Spark_Home as a string 

SPARK_HOME  = "C:\spark\spark-2.3.2-bin-hadoop2.7"

This resolves the error you had in your post. For the next error (you shared in the comment) it's similar to: The SPARK_HOME env variable is set but Jupyter Notebook doesn't see it. (Windows)

Skander HR
  • 580
  • 3
  • 14
  • after doing that, do I still need to use findspark.init()? – Micah Ondiwa Jun 08 '20 at 08:12
  • ValueError: Couldn't find Spark, make sure SPARK_HOME env is set or Spark is in an expected location (e.g. from homebrew installation). – Micah Ondiwa Jun 08 '20 at 08:13
  • So the first command was to resolve your first error. The second you need to check your file systems and your path to find the absolute path to your spark files. I updated my post because your error is similar to an old post. – Skander HR Jun 08 '20 at 08:21