0

I am using elasticsearch with jupyter python notebook in DSX. When I write a dataframe to objectstorage, I get an error:

ratings_df.write.format("org.elasticsearch.spark.sql").save("swift://DSConnections.spark/ratings.es")

Py4JJavaError: An error occurred while calling o96.save. : java.lang.ClassNotFoundException: Failed to find data source: org.elasticsearch.spark.sql. Please find packages at http://spark-packages.org

Community
  • 1
  • 1
Arun
  • 55
  • 5

1 Answers1

1

You need to install elasticsearch connector.

import pixiedust pixiedust.installPackage("org.elasticsearch:elasticsearch-spark_2.10:2.4.4")

Reference for PixiedustManager

http://datascience.ibm.com/docs/content/analyze-data/Package-Manager.html#installfrommavensearch

Thanks, Charles.

charles gomes
  • 2,145
  • 10
  • 15