0

"transforms": "TimestampConverter",

"transforms.TimestampConverter.type": "org.apache.kafka.connect.transforms.TimestampConverter$Value",
"transforms.TimestampConverter.format": "yyyy-MM-dd",
"transforms.TimestampConverter.target.type": "string",
"transforms.TimestampConverter.field" : "DateAtlevel2"

When this date filed is at parent level of JSON the SMT works fine , its not working when Date filed inside a JSOn array .

How to apply the SMT to convert a filed inside JSON array into a Date ?

I am using Kafka Mongo Sink connector .

Subash Mohapatra
  • 3
  • 1

1 Answers1

0

I assume that you are using the org.apache.kafka.connect.json.JsonConverter.java which is a schemaless converter. In this case, you would only be able to retrieve the level 1 fields.

Why? During the SMT phase, each record is stripped down to a plain Map<String, Object>, where String is the field name and the Object is its value. What the TimestampConverter does is to simply do a get on that map with the field name that you've passed as a config ( map.get("DateAtlevel2") ) which it can't be found.

To get access to level 2+ fields you would need some other type of input data that supports schemas ( AVRO, JSONSchema, or Protobuff ).

BogdanSucaciu
  • 884
  • 6
  • 13