2

Apache Toree is looking for the spark home directory (defaults to "/usr/local/spark", but when it can't find the directory due to spark having been installed via Homebrew, it throws an exception.

jupyter toree install

FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/spark/python/lib'

Where is the spark home when spark is installed via homebrew?

Garren S
  • 5,552
  • 3
  • 30
  • 45

2 Answers2

4

The directory Apache Toree is looking for when spark is installed via homebrew is in /usr/local/Cellar:

jupyter toree install --spark_home /usr/local/Cellar/apache-spark/2.1.0/libexec
/usr/local/Cellar/apache-spark/2.1.0/libexec/

It specifically wants the "libexec" directory where it can go into the "python/lib" sub-directory.

Garren S
  • 5,552
  • 3
  • 30
  • 45
1

If that doesn't work, you might additionally need to pass in a --user flag.

jupyter toree install --user --spark_home /usr/local/Cellar/apache-spark/2.2.0/libexec

Kinda like from this github issue, this jupyter documentation, and this other stack question.

kevcisme
  • 1,231
  • 1
  • 8
  • 9