0

When I input in command line :

pyspark --packages graphframes:graphframes:0.2.0-spark2.0-s_2.11

it will work well. But when I want to use ipython to launch my pyspark and use graphframes package, it doesn't work.

When I input in command line :

PYSPARK_DRIVER_PYTHON=ipython pyspark --packages graphframes:graphframes:0.2.0-spark2.0-s_2.11

the error is that: can't find module 'graphframes'

williamqzc
  • 3
  • 2

1 Answers1

0

Lack of Python 3 compatibility of the Spark packages is a known bug in graphframes 0.2.0 which has been resolved in 0.3.0. 

--packages graphframes:graphframes:0.3.0-spark2.0-s_2.11
user7337271
  • 1,662
  • 1
  • 14
  • 23