In my code I am have a requirement where I need to call spark sql for each of the rows of a dataset
.
Now, spark sql requires SparkSession
inside map function, which is not possible to pass as a broadcast Variable.
So, is there anyway to call Spark SQL inside a map function
?
I have checked online but I was not able to find any information related to same.
I am using Java as a Programming language for SPARK.
SPARK VERSION : 2.3.0
Thanks in advance.