0
root
 |-- code: string (nullable = true)
 |-- mnemonic: string (nullable = true)
 |-- key: long (nullable = true)
 |-- country: string (nullable = true)
 |-- crossID: struct (nullable = false)
 |    |-- codeKey: long (nullable = true)
 |    |-- codeGroupCode: string (nullable = true)
 |    |-- description: string (nullable = true)
 |    |-- externalCode: string (nullable = true)
 |-- ITEM: struct (nullable = false)
 |    |-- type: string (nullable = true)
 |    |-- indicator: string (nullable = true)
 |    |-- imageIndicator: string (nullable = true)
 |    |-- col20: struct (nullable = false)
 |    |    |-- cCode: string (nullable = true)
 |    |    |-- vCode: string (nullable = true)

The above is the schema of my dataframe. Here I would like group the crossID and its elements as list of maps. Please help me how to do it in java spark.

Faisal Ahamed R
  • 21
  • 6
  • 2
    Have you done literally anything? Not our job to do your coding. – basic May 15 '17 at 17:47
  • This is where I got stuck! Sqooped few tables, then converted them into dataframes, then joined(spark sql) them and used struct() on dataframe.select(). unable to proceed from here to create the structure that I want to. Later these will be pushed into elastic search repositories. Basically, would like to align the schema w.r.t elastic document. Thanks for your concern @basic. – Faisal Ahamed R May 15 '17 at 18:39

0 Answers0