I was trying to access the data inside object. The table is in pyspark dataframe, but some table values are in JSON format. I need t access the date from it and convert into meaningful format. Any help or idea will be great relief.
This is thing i'm working out:
I was able to extract data which are in array format using:
data_df=df_deidentifieddocuments_tst.select("_id",explode("annotationId").alias("annotationId")).select("_id","annotationId.*")
The above code doesn't work for date, as its showing type mismatch error
AnalysisException: cannot resolve 'explode(createdAt)' due to data type mismatch: input to function explode should be array or map type, not struct<$date:bigint>;
