8

I'm using the Confluent JDBCSourceConnector to read from an Oracle table. I am trying to use SMT to generate a key composed of 3 concatenated fields.

transforms=createKey
transforms.createKey.type=org.apache.kafka.connect.transforms.ValueToKey
transforms.createKey.fields=BUS_OFC_ID_CD,SO_TYPE,SO_NO

Using the above transformation, I get something like this:

{"BUS_OFC_ID_CD":"111","SO_TYPE":"I","SO_NO":"55555"}

I would like something like:

111I55555

Any idea on how I could concatenate the values only?

lloiacono
  • 4,714
  • 2
  • 30
  • 46
Alex
  • 45
  • 1
  • 7

1 Answers1

7

I could not resolve the issue above in the properties file. So, the work around was:

  • Create a view (we already had to do that to get mode=timestamp to work with our Oracle DB)
  • Add an extra field with the concatenated values for KEYNAME
  • Extract the concatenated value to use as the key.

For example:

CREATE VIEW XX_TEST_V AS 
SELECT BUS_OFC_ID_CD, SO_TYPE, SO_NO, BUS_OFC_ID_CD||SO_TYPE||SO_NO as KEYNAME 
FROM XX_TEST;

This would then give you a JSON key message of

{"KEYNAME ":"111I55555"}

To strip off the JSON to have just the text is then done in the properties file

For example:

transforms=createKey,extractString
transforms.createKey.type=org.apache.kafka.connect.transforms.ValueToKey
transforms.createKey.fields=KEYNAME
transforms.extractString.type=org.apache.kafka.connect.transforms.ExtractField$Key
transforms.extractString.field=KEYNAME

This should then give you the following as the key

"111I55555" 

Regards Peter

Peter Hollis
  • 71
  • 1
  • 2