0

I have a singleton object Main, and defined private var

private var params: Broadcast[Parameters] = _

Then I am initializing it through the main method

def main(args: Array[String]) = {
  ...
  params = spark.sparkContext.broadcast(new Parameters())

I have a method inside this object, then pass rdd argument and trying to get a value from broadcast variable in map transformation.

def testMethod(rdd: RDD[String]): Unit = {
  rdd.map(elem -> {
    val currentDate = params.value.CURRENT_DATE
    ...

Got an error on line where I try to get value grom params: NullPointerException. How to fix this code to make it work? I can't understand what is wrong with initialization.

J.J. Kubrick
  • 25
  • 4
  • When do you call `testMethod`? Is this after the initialization of `params`? What's the exception exactly? Edit the question to include the details. – Jacek Laskowski Mar 27 '18 at 16:58
  • It is Parameters class serialazable? If not you could try make it with @serializable annotation. Find [here](https://stackoverflow.com/questions/4067792/why-are-case-objects-serializable-and-case-classes-not) some details over serialization for scala classes or objects – abiratsis Mar 28 '18 at 11:09

0 Answers0