Using scala 2.12.17 I am trying to convert an Apache Flink 1.17.0 DataStream of Avro SpecificRecord
(org.apache.avro.specific.SpecificRecord) to a DataStream of Flink's RowData
(org.apache.flink.table.data.RowData) such that I can write the avro messages to storage using the Apache Flink Delta Connector (Apache Flink Delta Connector), which needs the RowData
format to be able to write. Additionally, I also need to get the associated RowType
(org.apache.flink.table.types.logical.RowType) of the RowData
to supply to the Delta Connector writer method.
I have tried to use AvroRowDataDeserializationSchema
(org.apache.flink.formats.avro.AvroRowDataDeserializationSchema) to do the conversion from a DataStream of SpecificRecord
to a DataStream of RowData
but I cannot wrap my head around how to do it in a generic way where I can do the conversion for any compiled avro schema of type SpecificRecord
and at the same time get the associated RowType
.