0

I'm trying to get Debezium Server running so that I can use GCP (Google) PubSub, and not have to use Kafka and the Kafka connectors. I have it mostly running, however, I'm having trouble with the using variables in the tranforms section to define a Topic name.

According to the documentation, when using the Outbox transformation, I can choose the Topic name by using the variable ${routedByValue} for the setting route.topic.replacement and this will use the value that is determined by the setting route.by.field. If the replacement setting is omitted, it will use a default topic name of outbox.event.<route.by.field value>.

When I try to use this variable in the 'application.properties' file ...

debezium.transforms.outbox.route.by.field=aggregate_type
debezium.transforms.outbox.route.topic.replace=${routedByValue}

... the Debezium Server stops with a NoSuchElementException, saying it cannot expand routedByValue. If I omit that setting, it works fine and defines the topic name as outbox.event.<route.by.field value>.

How can I use this variable correctly in the 'applications.properties' file so I can customise the topic name (e.g. route.topic.replace=myservice.${routedByValue})?

Dazfl
  • 627
  • 8
  • 23

2 Answers2

1

The way I got this to work was to do the following ...

debezium.transforms.outbox.route.by.file=aggregate_type
debezium.transforms.outbox.route.topic.replacement=$1

I believe this works because omitted from the config is another setting - debezium.transforms.outbox.route.topic.regex - and this has a default value of - (?<routedByValue>.*).

If I understand the documentation correctly, the $1 refers to the first group in the regex expression. In my case, this will return whatever the value of aggregrate_type equates to.

Dazfl
  • 627
  • 8
  • 23
0

I'm using Debezium Server 2.1 with Pulsar as sink type and the @Dazfl answer solve my issue !

debezium.transforms=outbox
debezium.transforms.outbox.type=io.debezium.transforms.outbox.EventRouter
debezium.transforms.outbox.route.topic.replacement=outbox.event.transactions.$1

Although the Debezium Server docs says to use $routedByValue, this do not works as expeceted...

chseki
  • 1