0

The structure what I have:

{
 "person":"Jhon",
 [
    {"model":"golf","mark":"vw","fuel":"d","year":2000},
    {"model":"yamaha r1","year":1998,"abs":true}
 ]
}
//
StructType(
  StructField("person", StringPrimitive, false),
  ArrayType( /* ???? */ ) // Should be possible to define several structures
)

So, the Idea is simple. Some person has array of his own transport and this transport can be of type: CarEntity, MotorcycleEntity, BicycleEntity, AirplaneEntity and so on...

The second situation when I receive only one transport entity for each person. Smth like this:

{"person":"Jhon",{"model":"yamaha r1","year":1998,"abs":true}}
//
StructType(
  StructField("person", StringPrimitive, false),
  StructField( /* ???? */ ) // Should be possible to define several structures
)

And in this case the transport can be one of previously mentioned types.

What is the right way to build StructType for that?

How to define ArrayType of multiple types or multiple StructType types?

Aleksandrs Rudzitis
  • 661
  • 4
  • 9
  • 30

1 Answers1

0

If you are using Spark 1.x you could try to achieve the same result with an UDT (as described in this topic). But for Spark 2.x this functionality has been removed.

RudyVerboven
  • 1,204
  • 1
  • 14
  • 31