0

Im trying to install kafka-connect-zeebee plugin on my self managed confluent container running on eks.

I’m able to kubectl exec into the connect-0 container and confluent-hub is installed.

How do i install connector plugins on a confluent cluster running in eks?

  1. cp kafka-connect-zeebee jar to /tmp directory of container and try copy that jar to /usr/share/confluent-hub-components/ then i get - permission denied
  2. install any connector via confluent-hub then i get - Failed to create output directory: /usr/share/confluent-hub-components/confluentinc.
OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Socrates Hlapolosa
  • 3
  • 1

1 Answers1

0

You need to create your Docker container with a RUN confluent-hub install command (or COPY kafka-connect-zeebee.jar, but not to /tmp), from which you build and deploy to your EKS cluster

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
  • thanks for the speedy response @OneCricketeer. but just some questions; i've created my confluent cluster using, `helm upgrade --install confluent-operator confluentinc/confluent-for-kubernetes kubectl apply -f https://raw.githubusercontent.com/confluentinc/confluent-kubernetes-examples/master/quickstart-deploy/confluent-platform.yaml ` and this creates a connect pod and all this running on eks. it is this pod that i cannot copy the zeebee jar to. so are you saying i should create another pod? i can ssh to it, but cant run confluent-hub install nor copy the jar. – Socrates Hlapolosa Mar 07 '23 at 11:28
  • No exec, and no SSH. Use `--values` to override the Connect spec `application.image`, after you've built and pushed your own image to a Docker Registry containing the plugins you need https://docs.confluent.io/kafka-connectors/self-managed/extending.html#create-a-docker-image-containing-local-connectors – OneCricketeer Mar 07 '23 at 16:29
  • 1
    makes perfect sense. the key is not to try modify the existing connect pod, but to create my own image with the connector (whether confluent-hub or COPY jar) then deploy that into the connect cluster instead – Socrates Hlapolosa Mar 08 '23 at 07:35