1

I have a Confluence Kafka consumer written in Golang. I am trying to deploy it in a PKS cluster.

The Kafka config looks like this,

kafka.bootstrap.servers=server.myserver.com
kafka.security.protocol=SASL_SSL
kafka.sasl.mechanisms=GSSAPI
kafka.group.id=kafka-go-getting-started
kafka.auto.offset.reset=latest
kafka.topic=topic.consumer-topic
acks=all

I need to configure my Dockerfile for GSSAPI mechanism with SASL_SSL protocol. I have managed to resolve the GSSAPI thing, however, currently it shows,

**Failed to create consumer: Unsupported value "SASL_SSL" for configuration property "security.protocol": OpenSSL not available at build time**

Here is how my Dockerfile looks like:

FROM golang:1.19-alpine3.16 as c-bindings

RUN apk update && apk upgrade && apk add pkgconf git bash build-base sudo


RUN git clone https://github.com/edenhill/librdkafka.git
RUN cd librdkafka && ./configure && make && sudo make install


FROM c-bindings as app-builder

WORKDIR /go/app


COPY . .

RUN go mod download
RUN go mod verify


RUN go build -race -tags musl --ldflags "-extldflags -static -s -w" -o main ./main.go

FROM scratch AS app-runner

WORKDIR /go/app/

COPY --from=app-builder /go/app/main ./main


CMD ["/go/app/main"]`

Tried some ways in Dockerfile to make OpenSSL available, however things are stuck at same. Not sure if both GSSAPI mechanism as well as SASL_SSL protocol can be resolved over a common solution.

[Dec 05, 2022] Latest try:

Dockerfile,


FROM golang:1.19-alpine as c-bindings

RUN apk update && apk upgrade && apk add pkgconf git bash build-base sudo

FROM c-bindings as app-builder

WORKDIR /go/app

COPY . .

RUN go mod download
RUN go mod verify

RUN apk add zstd-dev

RUN apk add krb5
RUN apk add cyrus-sasl-gssapiv2
RUN apk add cyrus-sasl-dev

RUN apk add openssl-dev


RUN git clone https://github.com/edenhill/librdkafka.git
RUN cd librdkafka && ./configure --install-deps && make && sudo make install

COPY krb5.conf /etc/krb5.conf
COPY jaas.conf /etc/jaas.conf

RUN go build -race -tags dynamic -o main ./main.go


CMD ["/go/app/main"]

Kafka config -

kafka.bootstrap.servers=server.myserver.com
kafka.security.protocol=SASL_SSL
kafka.sasl.mechanism=GSSAPI
kafka.group.id=kafka-go-getting-started
kafka.auto.offset.reset=latest
kafka.topic=topic.consumer-topic
kafka.ssl.ca.location=/etc/ssl/certs/my-cert.pem
kafka.sasl.kerberos.service.name=kafka
kafka.sasl.kerberos.keytab=/etc/security/keytab/consumer.keytab
kafka.sasl.kerberos.principal=principal@myprincipal.COM
acks=all

Now the container is technically running. However, it is not able to run the Kafka consumer application with below errors -

GSSAPI Error: A token had an invalid MIC (unknown mech-code 0 for mech unknown)

Pramit Pakhira
  • 135
  • 1
  • 8

1 Answers1

0

that is because you are missing the SSL or SASL dependancies you would need to make sure that libssl-dev, hoewever it could also needs those libsasl2-dev, libsasl2-modules, but libssl-dev should be enough though

adding the following to the DOCKERFILE should help to resolve it

RUN apk add libressl-dev

here is the official libssl and the alpine pkg

mooga
  • 3,136
  • 4
  • 23
  • 38
  • still it does not work. Added it just below librdkafka installation. Also, tried with this, RUN apk add --update openssl && \ rm -rf /var/cache/apk/* No luck. :( – Pramit Pakhira Nov 28 '22 at 15:46
  • is it the same error ? or it does return something different ? – mooga Dec 05 '22 at 09:30
  • That time it was the same error only. After that, I tried with several other ways and currently my configurations look like above. This time I am getting a different error while the container is running (please see above update, [Nov 05, 2022] Latest try). – Pramit Pakhira Dec 05 '22 at 10:55
  • do you have KDC server running ? I mean the reason for that error obviously is `kafka.sasl.kerberos` – mooga Dec 05 '22 at 14:52
  • will it be required to have KDC server configured in my side if I am to consume only? – Pramit Pakhira Dec 09 '22 at 08:03