Confluent Kafka Dotnet Kerberos Support Dockerfile (No provider for SASL mechanism GSSAPI)

Question

I am trying to create a dockerfile which is a kafka client using confluent-kafka-dotnet. It has to use Kerberos keytabs to connect, and therefore I have read this Github wiki.

Here is my dockerfile:

# ---- dotnet build stage ----
FROM mcr.microsoft.com/dotnet/core/sdk:3.1 as build

ARG BUILDCONFIG=RELEASE
ARG VERSION=1.0.0

# Installing dependencies for li
RUN apt-get update && apt-get install libsasl2-modules-gssapi-mit libsasl2-dev unzip build-essential -y

COPY ./lib/ /
RUN unzip librdkafka-1.4.4.zip && \
    cd librdkafka-1.4.4 && \
    ./configure && \
    make && \
    make install

WORKDIR /build/

COPY ./DashboardServer/DashboardServer.csproj ./DashboardServer.csproj
RUN dotnet nuget add source https://ci.appveyor.com/nuget/docker-dotnet-hojfmn6hoed7 && \
    dotnet restore ./DashboardServer.csproj

COPY ./DashboardServer ./

RUN dotnet build && dotnet publish ./DashboardServer.csproj -c ${BUILDCONFIG} -o out /p:Version=${VERSION}

# ---- final stage ----

FROM ubuntu:20.04

LABEL Maintainer=""

ENV PROGRAM_HOME=/opt/DashboardServer
ENV ASPNETCORE_ENVIRONMENT=Production

RUN apt-get update && \
    apt-get install -y wget && wget https://packages.microsoft.com/config/ubuntu/19.10/packages-microsoft-prod.deb -O packages-microsoft-prod.deb && \
    dpkg --purge packages-microsoft-prod && dpkg -i packages-microsoft-prod.deb && \
    apt-get update && \
    apt-get install aspnetcore-runtime-3.1 curl -y

RUN export DEBIAN_FRONTEND=noninteractive && apt-get install libsasl2-modules-gssapi-mit libsasl2-dev -y krb5-user

# Kafka SASL directory (keytab is placed here)
RUN mkdir /sasl/

ENV KEYTAB_LOCATION=/sasl/dashboards.service.keytab

COPY --from=build /build/out ${PROGRAM_HOME}

# Copy necessary scripts + configuration
COPY scripts /tmp/
RUN chmod +x /tmp/*.sh && \
    mv /tmp/* /usr/bin && \
    rm -rf /tmp/*

CMD [ "docker-entrypoint.sh" ]

My C# consumer config code is as follows:

var consumerConfig = new ConsumerConfig {
                GroupId = "command-server" + KafkaHelpers.Servername,
                BootstrapServers = KafkaHelpers.BootstrapServers,
                AutoOffsetReset = AutoOffsetReset.Latest,
                SecurityProtocol = SecurityProtocol.SaslPlaintext,
                SaslKerberosServiceName = "kafka",
                SaslKerberosKeytab = Environment.GetEnvironmentVariable("KEYTAB_LOCATION"),
                SaslKerberosPrincipal = "dashboardserver/<<IPAddress>>"
            };

But I get the following exception when I start my client

Unhandled exception. Unhandled exception. System.InvalidOperationException: No provider for SASL 
mechanism GSSAPI: recompile librdkafka with libsasl2 or openssl support. Current build options: PLAIN SASL_SCRAM OAUTHBEARER

Can anyone help me out or point me in the right direction? I found this on GitHub But I can't seem to get it to work.

I don't know how to install librdkafka step by step.

I'm facing similar problem with .Net 6 Confluent.Kafka, When you say COPY ./lib/ . where are you getting the ./lib content — Elias Ghali, Apr 05 '22 at 09:47

score 0 · Answer 1 · answered Jul 22 '20 at 06:34

0

confluent-kafka-dotnet depends on librdkafka.redist from where it loads librdkafka by default. The builds in librdkafka.redist lacks SASL/GSSAPI support for most platforms (due to dependency issues with libsasl2 and all its support libs), so your approach of building your own librdkafka version is the proper one, but you will need to tell confluent-kafka-dotnet to load your build of librdkafka rather than the librdkafka.redist ones, you do this by calling

Confluent.Kafka.Library.Load("/usr/local/lib/librdkafka.so");

before calling any other confluent-kafka-dotnet API.

answered Jul 22 '20 at 06:34

Edenhill

2,897
22
35

Hey! Thank you for the reply. When I try I get `Unhandled exception. System.DllNotFoundException: Failed to load the librdkafka native library.`. It seems like a dependency problem, is there any other dependencies than libsasl2-modules-gssapi-mit and libsasl2-dev? I can see when I manually install librdkafka it has a lot of "failed" when it checks. – Wiklo Jul 23 '20 at 08:26
1

Try `ldd path/to/librdkafka.so` in your container to find out if there are any missing dependencies. – Edenhill Aug 17 '20 at 08:03

Confluent Kafka Dotnet Kerberos Support Dockerfile (No provider for SASL mechanism GSSAPI)

1 Answers1

Linked