I have a dataframe with below schema
|-- matches: array (nullable = true)
| |-- element: struct (containsNull = true)
| | |-- influencer: string (nullable = true)
| | |-- frequency: long (nullable = true)
| | |-- relevance: double (nullable = true)
I need to convert matches column to json format, but having difficulty is iterating struct elements.
Tried writing a udf to convert it to json, but unable to process ArrayElement. Can someone please suggest me a way to do it.
def convertToJSON = udf((value: Array[ArrayElement]) => {
Json.obj(
"matches" -> value.map { x => Json.obj(
"influencer" -> "",
"overlap" -> "",
"relevance" -> ""
)
}).toString()
})