Asp.net core API server logs events to Confluent Cloud Kafka when testing locally, but not when hosting on Azure App Service

Question

I have some code that my Asp.net Core Web API uses to log certain events to a Kafka server running in Confluent Cloud. When I run the API server on my local machine, it can send and receive with Kafka just fine, but when it is running on an Azure App Service, I receive "Local: Message Timed Out" errors. Is there something about Azure App Service networking that I can modify to make the Kafka network traffic flow correctly?

Here is a snippet of the code below:

    public class ConfluentKafkaService {
        private readonly ClientConfig clientConfig = new ClientConfig
        {
            BootstrapServers = "...",
            ClientId = Dns.GetHostName(),
            SecurityProtocol = SecurityProtocol.SaslSsl,
            SaslMechanism = SaslMechanism.Plain,
            SaslUsername = "...",
            SaslPassword = @"..."
        };

        public async Task SendDeviceEvent(DeviceEvent de) {
            var config = new ProducerConfig(clientConfig);
            string topicName = $"...";

            using var producer = new ProducerBuilder<Null, DeviceEvent>(config)
                .Build();
            try {
                await producer.ProduceAsync(topicName, new Message<Null, DeviceEvent> { Value = de });
            }
            catch (ProduceException<Null, string> e) {
                Console.WriteLine($"Error producing message: {e.Message}");
            }
        }
    }

score 0 · Accepted Answer · answered May 07 '20 at 18:16

My connectivity issue was ultimately caused because Azure App Service does not expose its Trusted Certificate store to librdkafka correctly. I downloaded cacert.pem from https://curl.haxx.se/docs/caextract.html and pointed to it by setting SslCaLocation in my ClientConfig like so:

private readonly ClientConfig clientConfig = new ClientConfig
{
    BootstrapServers = "",
    ClientId = Dns.GetHostName(),
    SecurityProtocol = SecurityProtocol.SaslSsl,
    SslCaLocation = Path.Combine("assets", "cacert.pem"),
    SaslMechanism = SaslMechanism.Plain,
    SaslUsername = ""
    SaslPassword = ""
}

For further information, see this issue: https://github.com/confluentinc/confluent-kafka-dotnet/issues/1112

Asp.net core API server logs events to Confluent Cloud Kafka when testing locally, but not when hosting on Azure App Service

1 Answers1