Add jar to general Kafka Connect classpath in the Confluent Docker

Question

I am using the confluent docker for Kafka connect v4.1.1 and would like to add a jar with specific log4j appenders.
With the classpath isolation by connector, I am not sure where to place that jar in the confluent docker, as it is used by the parent kafka connect itself and not the connectors.

Thanks in advance for the help!

What happens when you just pick any directory for the `plugin.path`? — OneCricketeer, Jun 15 '18 at 11:30

score 8 · Accepted Answer · edited Jun 20 '20 at 09:12

8

If you want to add a jar that will be used by the Connect framework and is not specific to any of the connectors, you have the following options:

Add the jar to the classpath with:

-e CLASSPATH=/path/to/custom/library.jar
Copy the jar directly to the directory that includes the Kafka Connect framework. In Confluent docker images this is currently:

/usr/share/java/kafka

You need to make sure this jar is not in conflict with any of the dependencies of Kafka Connect.

edited Jun 20 '20 at 09:12

Community

1
1

answered Jun 15 '18 at 14:37

Konstantine Karantasis

1,855
14
15

I remember trying no.2 but without success, will see if I did anything wrong and also give no.1 a shot. Thanks! – Cherif Naggar Jun 17 '18 at 00:33
2

`-e CLASSPATH=/path/to/custom/library.jar` goes where? – Sherms Jan 28 '22 at 19:38

Javier Montón · Answer 2 · 2023-07-13T14:00:19.440

For me, it didn't work on /usr/share/java/ and for what I've seen, the Kafka Connect Docker uses a different path in Standalone than in Distributed mode.

Running something like ps aux | grep java in your running instance will show you the paths it's loading, you'll see something like: java -cp /usr/share/java/cp-base-new/*

In Standalone it's using /usr/share/java/cp-base-new/

In Distributed mode it uses more paths, one of them is /etc/kafka-connect/jars/

Add jar to general Kafka Connect classpath in the Confluent Docker

2 Answers2