Getting below exception while creating key out of one field coming in "value" part of payload

Question

SMT is not working with below configuration, any idea

"transforms": "createKey",
"transforms.createKey.type":"org.apache.kafka.connect.transforms.ValueToKey",
"transforms.createKey.fields":"column_id",
getting below exception

Exception:    org.apache.kafka.connect.errors.ConnectException: Tolerance exceeded in error handler
    at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:178)
    at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execute(RetryWithToleranceOperator.java:104)
    at org.apache.kafka.connect.runtime.TransformationChain.apply(TransformationChain.java:50)
    at org.apache.kafka.connect.runtime.WorkerSourceTask.sendRecords(WorkerSourceTask.java:292)
    at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:228)
    at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:175)
    at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:219)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
    Caused by: java.lang.NullPointerException
    at org.apache.kafka.connect.transforms.ValueToKey.applyWithSchema(ValueToKey.java:85)
    at org.apache.kafka.connect.transforms.ValueToKey.apply(ValueToKey.java:65)
    at org.apache.kafka.connect.runtime.TransformationChain.lambda$apply$0(TransformationChain.java:50)
    at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndRetry(RetryWithToleranceOperator.java:128)
    at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:162)

Added configuration in kafka-connect as well: key.converter.schemas.enable=true value.converter.schemas.enable=true, to enable the schema needed by class ValueToKey

schema is not coming in the kafka topic only payload is coming, I am using debezium connector as a source — Girish, Jul 01 '19 at 06:15
@cricket_007, schema is not coming in the kafka topic only payload is coming, I am using debezium connector as a source — Girish, Jul 01 '19 at 06:24

score 0 · Answer 1 · answered Jan 31 '20 at 11:32

This might be late however for me the problem was case sensitive. My column name was ID and in configuration it was id. So below is configuration that worked for me

transforms=createKey
transforms.createKey.fields=ID
transforms.createKey.type=org.apache.kafka.connect.transforms.ValueToKey

Getting below exception while creating key out of one field coming in "value" part of payload

1 Answers1