Spring Cloud Stream [2021.0.5] Kafka Batch mode Avro native encoding doesn't work with spring cloud sleuth

Question

i'm working on upgrading spring boot to 2.7.8 and spring cloud to 2021.0.5.

I have Spring cloud stream kafka consumer using avro deserialization in batch-mode, and I was trying to use useNativeEncoding according to documentation.

the problem is when using an input of Message<List> the spring cloud stream code overrides (when using sleuth) the flag of native encoding to false in this class SimpleFunctionRegistry, this the message payload is empty.

without using the Message> it works fine, i.e. List.

after spending more than one day trying to debug the problem without understanding why, I took it to a side project to test it, and it stopped working after using sleuth.

The Bug

the problem is one the class SimpleFunctionRegistry on methodprivate FunctionInvocationWrapper wrapInAroundAdviceIfNecessary(FunctionInvocationWrapper function) it calls the apply and override the flag

spring cloud stream team is there any workaround? or an easy fix?

application.yaml example

spring:
  cloud:
    stream:
      binders:
        kafka-string-avro-native:
          type: kafka
          defaultCandidate: true
          environment.spring.cloud.stream.kafka.binder.consumerProperties:
            dlqProducerProperties.configuration.key.serializer: org.apache.kafka.common.serialization.StringSerializer
            dlqProducerProperties.configuration.value.serializer: io.confluent.kafka.serializers.KafkaAvroSerializer
            key.deserializer: org.apache.kafka.common.serialization.StringDeserializer
            value.deserializer: io.confluent.kafka.serializers.KafkaAvroDeserializer
            schema.registry.url: ${SCHEMA_REGISTRY_URL:http://0.0.0.0:55013}
            specific.avro.reader: true
            useNativeDecoding: true

      bindings:
        revenueEventConsumer-in-0:
          binder: kafka-string-avro-native
          destination: email.campaign_revenue_events
          group: test-4
          consumer:
            concurrency: 1
            batch-mode: true
            use-native-decoding: true
      function:
        definition: revenueEventConsumer
      kafka:
        binder:
          brokers: 0.0.0.0:55008

score 0 · Answer 1 · answered Feb 12 '23 at 08:40

i found a workaround for the issue by overriding the Bean TraceFunctionAroundWrapper and overriding the setSkipInputConversion(true)

see code below

@Bean
    @Primary
    TraceFunctionAroundWrapper customTraceFunctionAroundWrapper(Environment environment, Tracer tracer, Propagator propagator,
                                                          Propagator.Setter<MessageHeaderAccessor> injector, Propagator.Getter<MessageHeaderAccessor> extractor,
                                                          ObjectProvider<List<FunctionMessageSpanCustomizer>> customizers) {
        return new CustomTraceFunctionAroundWrapper(environment, tracer, propagator, injector, extractor,
            customizers.getIfAvailable(ArrayList::new));
    }

public class CustomTraceFunctionAroundWrapper extends TraceFunctionAroundWrapper {
    public CustomTraceFunctionAroundWrapper(Environment environment, Tracer tracer,
                                            Propagator propagator,
                                            Propagator.Setter<MessageHeaderAccessor> injector,
                                            Propagator.Getter<MessageHeaderAccessor> extractor) {
        super(environment, tracer, propagator, injector, extractor);
    }

    public CustomTraceFunctionAroundWrapper(Environment environment, Tracer tracer, Propagator propagator, Propagator.Setter<MessageHeaderAccessor> injector,
                                            Propagator.Getter<MessageHeaderAccessor> extractor,
                                            List<FunctionMessageSpanCustomizer> customizers) {
        super(environment, tracer, propagator, injector, extractor, customizers);
    }

    @Override
    protected Object doApply(Object message, SimpleFunctionRegistry.FunctionInvocationWrapper targetFunction) {
        targetFunction.setSkipInputConversion(true);
        return super.doApply(message, targetFunction);
    }
}

this is only a workaround until the bug is fixed is spring cloud stream and sleuth

Spring Cloud Stream [2021.0.5] Kafka Batch mode Avro native encoding doesn't work with spring cloud sleuth

1 Answers1