How to use hiveContext in streaming from executors?

Question

I'm trying to use an instance of HiveContext in a Spark streaming application (1.6), but it fails with the following exception:

java.lang.NullPointerException at org.apache.spark.sql.SQLConf.getConf(SQLConf.scala:638)

In particolar, I want to create a dataframe and execute a query for each processed message.

Someone can help me?

Answer 1

This error appears when you try to use SQLContext or HiveContext directly from executors.

(Spark 1.6)