Maintaining the original traceId while passing messages through Kafka with quarkus and opentracing

Question

I'm trying to create the most basic working example with two Quarkus (2.4.1.Final) microservices (a producer and a consumer) that communicate through Kafka and are traced with opentracing.

I have followed the kafka and the opentracing tutorial, ran the producer and consumer in dev mode (so they create that redpanda kafka broker), and then attempted to emit a POJO and log the traceId in both the consumer and producer. As far as I understand, this should work out of the box.

The POJO is sent, serialized and de-serialized without a hitch. The kafka message header that the consumer receives even has the correct original trace and span id injected into it (I've checked with debugging both the producer and consumer) using that uber-trace-id field.

But, for some reason, when logging the trace id, they don't match. It's like the tracing context "forgets" about the span it receives via kafka. Note that my business need is to just print the traceId every log so we can follow printed logs through kibana.

The producer:

package com.example;

import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;

import org.eclipse.microprofile.reactive.messaging.Channel;
import org.eclipse.microprofile.reactive.messaging.Emitter;
import org.jboss.logging.Logger;

@Path("/hello")
public class ExampleResource {
    private static final  Logger log = Logger.getLogger(ExampleResource.class);
    private final Emitter<Person> peopleEmitter;

    public ExampleResource(@Channel("people") Emitter<Person> peopleEmitter) {this.peopleEmitter = peopleEmitter;}

    @GET
    @Path("/{name}")
    @Produces(MediaType.APPLICATION_JSON)
    public Person hello(@PathParam("name") String name) {
        var p = new Person();
        p.name = name;
        log.info("Produced " + p.name);
        peopleEmitter.send(p);
        return p;
    }
}

The consumer:

package com.example;

import org.eclipse.microprofile.opentracing.Traced;
import org.eclipse.microprofile.reactive.messaging.Incoming;
import org.jboss.logging.Logger;

import javax.enterprise.context.ApplicationScoped;

@ApplicationScoped
public class PeopleConsumer {
    private static final Logger log = Logger.getLogger(PeopleConsumer.class);

    @Traced
    @Incoming("people")
    public void process(Person person) {
        log.info("received " + person.name);
    }
}

The POJO:

package com.example;

public class Person {
    public String name;
}

App config for the producer:

quarkus.application.name=producer
quarkus.http.port=8090
quarkus.log.console.format=%d{HH:mm:ss} %-5p traceId=%X{traceId}, spanId=%X{spanId}, [%c{2.}] (%t) %s%e%n

mp.messaging.outgoing.people.connector=smallrye-kafka
mp.messaging.outgoing.people.interceptor.classes=io.opentracing.contrib.kafka.TracingProducerInterceptor

And the consumer:

quarkus.http.port=8091
quarkus.application.name=consumer
quarkus.log.console.format=%d{HH:mm:ss} %-5p traceId=%X{traceId}, spanId=%X{spanId}, [%c{2.}] (%t) %s%e%n

mp.messaging.incoming.people.connector=smallrye-kafka
mp.messaging.incoming.people.interceptor.classes=io.opentracing.contrib.kafka.TracingConsumerInterceptor

Serializer/deserializer

public class PersonDeserializer extends ObjectMapperDeserializer<Person> {
    public PersonDeserializer() {
        super(Person.class);
    }
}


public class PersonSerializer extends ObjectMapperSerializer<Person> {
}

The dependencies (same for both):

    <dependencies>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-smallrye-opentracing</artifactId>
        </dependency>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-smallrye-reactive-messaging-kafka</artifactId>
        </dependency>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-smallrye-context-propagation</artifactId>
        </dependency>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-arc</artifactId>
        </dependency>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-resteasy-reactive-jackson</artifactId>
        </dependency>
        <dependency>
            <groupId>io.opentracing.contrib</groupId>
            <artifactId>opentracing-kafka-client</artifactId>
        </dependency>
    </dependencies>

Basic use case:

Put http://localhost:8090/hello/John into a browser. You will see log at the producer:

18:43:24 INFO  traceId=00019292a43349df, spanId=19292a43349df, [co.ex.ExampleResource] (executor-thread-0) Produced John

And at the consumer

18:43:25 INFO  traceId=1756e0a24c740fa6, spanId=1756e0a24c740fa6, [co.ex.PeopleConsumer] (pool-1-thread-1) received John

Notice the trace ids being different. I am not sure what else I'm supposed to be doing/configuring...