+--------------------+--------------------+
| _VALUE| paraarr|
+--------------------+--------------------+
|Archer, Edward Pa...|[, [[, Arbitrator...|
|Archer, Edward Pa...|[, [[, Member:],,...|
|Archer, Edward Pa...|[, [[, Experience...|
|Archer, Edward Pa...|[, [[, Publicatio...|
|Belcher, A. Lee (...|[, [[, Arbitrator...|
|Belcher, A. Lee (...|[, [[, Member:],,...|
|Belcher, A. Lee (...|[, [[, Experience...|
|Bloodsworth, Davi...|[, [[, Arbitrator...|
|Bloodsworth, Davi...|[, [[, Member:],,...|
|Bloodsworth, Davi...|[, [[, Experience...|
|Bloodsworth, Davi...|[, [[, Public Sec...|
|Bloodsworth, Davi...|[, [[, Issue:],,,,]]|
|Bloodsworth, Davi...|[, [[, Industry:]...|
|Brent, Daniel F. ...|[, [[, Arbitrator...|
|Brent, Daniel F. ...|[, [[, Profession...|
|Brent, Daniel F. ...|[, [[, Arbitratio...|
|Brent, Daniel F. ...|[, [[, Permanent ...|
|Brent, Daniel F. ...|[, [[, Issues:],,...|
|Brent, Daniel F. ...|[, [[, Industries...|
|Chiesa, Mario (Mi...|[, [[, Arbitrator...|
+--------------------+--------------------+Using AWS glue sql context , i want to explore the paraar column and want to extract data from this column. I am very new to AWS glue.I am not able to find how should i go about it.
I have executed below code to get this data column.
from pyspark.sql.functions import *
dyf1.toDF().createOrReplaceTempView("x3")
df1=spark.sql("select p._VALUE,explode(para_array) as paraarr from x2")
