3

Using the new function based programming model of Spring Cloud Stream I need to have access to some of the headers of the message I consume. I figured I would consume Message<MyDto> instead of only MyDto, but if I do this MyDto properties are all null.

This is in broad strokes what I want to do :

@ToString
@Getter
@Setter
public class MyDto {
   private OtherDto otherDto;
}

@Bean
public Consumer<Message<MyDto>> onRawImport() {
    message -> logger.info("Received {}", message.getPayload());  // <-- prints "Received MyDto(otherDto=OtherDto(...))"
}

whereas the following works perfectly in my configuration

@Bean
public Consumer<MyDto> onRawImport() {
    myDto -> logger.info("Received {}", myDto);  // <-- "Received MyDto(otherDto=null)"
}

Is there a simple way to consume Message directly ?


Addendum :

If I turn DEBUG on for org.springframework.cloud.function.context.catalog i see this with Consumer<MyDto> :

BeanFactoryAwareFunctionRegistry : Applying function: onRawImport
BeanFactoryAwareFunctionRegistry : Applying type conversion on input value GenericMessage [payload=byte[288], headers=...snip...]
BeanFactoryAwareFunctionRegistry : Function type: java.util.function.Consumer<MyDto>
BeanFactoryAwareFunctionRegistry : Raw type of value: GenericMessage [payload=byte[288], ...snip...}] is class MyDto
BeanFactoryAwareFunctionRegistry : Converted from Message: MyDto(otherDto=OtherDto(...))
BeanFactoryAwareFunctionRegistry : Converted input value MyDto(otherDto=OtherDto(...))
MyOwnListener : Received MyDto(id=5, message=test)

and this with Consumer<Message<MyDto>>

BeanFactoryAwareFunctionRegistry : Applying function: onRawImport
BeanFactoryAwareFunctionRegistry : Applying type conversion on input value GenericMessage [payload=byte[288], headers=...snip...]
BeanFactoryAwareFunctionRegistry : Function type: Function type: java.util.function.Consumer<org.springframework.messaging.Message<MyDto>>
BeanFactoryAwareFunctionRegistry : Raw type of value: GenericMessage [payload=byte[288], ...snip...}] is class MyDto
BeanFactoryAwareFunctionRegistry : Converted from Message: MyDto(otherDto 
=null)
BeanFactoryAwareFunctionRegistry : Converted input value MyDto(otherDto 
=null)
MyOwnListener : Received MyDto(otherDto 
=null)
Sébastien Nussbaumer
  • 6,202
  • 5
  • 40
  • 58
  • That is interesting. `message.getPayload()` should print the values. Are you sure the message is created properly on the producing side? – sobychacko Apr 07 '21 at 17:21
  • 1
    quite sure yes. I launched my microservice with Consumer> and each message consumption failed with a NullPointerException when accessing the dto fields, then changed to Consumer + reset kafka offsets + restart microservice and it worked. I wanted to know if consuming Message rather than MyDto was a supported feature and how to do it. You seem to say this should work, so I'll continue looking into it and let you know what I find. – Sébastien Nussbaumer Apr 08 '21 at 06:45

1 Answers1

1

Which version are you using? I just tested it with Boot 2.4.4, cloud 2020.0.2 and it works fine...

@SpringBootApplication
public class So66990612Application {

    public static void main(String[] args) {
        SpringApplication.run(So66990612Application.class, args);
    }

    @Bean
    Consumer<Message<Foo>> input() {
        return System.out::println;
    }


    @Bean
    public ApplicationRunner runner(RabbitTemplate template) {
        return args -> {
            template.convertAndSend("input-in-0", "x", "{\"bar\":\"baz\"}",
                    msg -> {
                        msg.getMessageProperties().setContentType("application/json");
                        return msg;
                    });
        };
    }

    public static class Foo {

        private String bar;

        public String getBar() {
            return this.bar;
        }

        public void setBar(String bar) {
            this.bar = bar;
        }

        @Override
        public String toString() {
            return "Foo [bar=" + this.bar + "]";
        }

    }

}
GenericMessage [payload=Foo [bar=baz], headers={amqp_receivedDeliveryMode=PERSISTENT, amqp_receivedExchange=input-in-0, amqp_deliveryTag=1, deliveryAttempt=1, amqp_consumerQueue=input-in-0.anonymous.Uh_s89lKRnKeJ3ls991pXA, amqp_redelivered=false, amqp_receivedRoutingKey=x, amqp_contentEncoding=UTF-8, id=1a848cf6-3f85-c017-fa70-d52f43c0fc67, amqp_consumerTag=amq.ctag-w6ZyXBGOtC-q7rCY8Jy-gA, sourceData=(Body:'{"bar":"baz"}' MessageProperties [headers={}, contentType=application/json, contentEncoding=UTF-8, contentLength=0, receivedDeliveryMode=PERSISTENT, priority=0, redelivered=false, receivedExchange=input-in-0, receivedRoutingKey=x, deliveryTag=1, consumerTag=amq.ctag-w6ZyXBGOtC-q7rCY8Jy-gA, consumerQueue=input-in-0.anonymous.Uh_s89lKRnKeJ3ls991pXA]), contentType=application/json, timestamp=1617888626028}]
Gary Russell
  • 166,535
  • 14
  • 146
  • 179
  • I am on Boot 2.3.8 + Cloud Hoxton.SR10. We plan on migrating to 2020.0.2 in a few months. It works as soon as I set conversionHint to null in ApplicationJsonMessageMarshallingConverter just before the call to super.convertFromInternal: ``` if (targetClass.equals(hint)) { conversionHint = null; } ``` Maybe this is has been corrected in 2020.0.x – Sébastien Nussbaumer Apr 08 '21 at 13:51
  • Hmmm - I just tried it with 2.3.9, Hoxton.SR10, and it still works for me (with both the RabbitMQ and Kafka binders). – Gary Russell Apr 08 '21 at 13:59
  • Just tried wiht boot 2.3.9 and same pb. I must be missing a small piece somewhere. As I said, when the conversionHint is null in ApplicationJsonMessageMarshallingConverter then everything works as advertised ... that hint result in a call to "objectMapper.readerWithView(view).forType(javaType).readValue((byte[]) payload)" in MappingJackson2MessageConverter. Without the hint the a simple objectMapper.readValue((byte[]) payload, javaType) is done and works. Maybe my objectMapper is the culprit ... I'll try to look into that. – Sébastien Nussbaumer Apr 08 '21 at 15:43
  • Seems to be a good lead, If I create an ApplicationJsonMessageMarshallingConverter with an ObjectMapper with DEFAULT_VIEW_INCLUSION set to true, then deserialization of my Message works.... I have to stop for today, tomorrow will bring more clues ;) – Sébastien Nussbaumer Apr 08 '21 at 16:14
  • hum, could be related to https://github.com/spring-cloud/spring-cloud-function/issues/608#issuecomment-799527590, in SimpleFunctionRegistry.convertInputValueIfNecessary we have a different behaviour when we consume Message and MyDto : convertWithHint is set to true at line 828 because rawType.equals(type) is false in that case – Sébastien Nussbaumer Apr 09 '21 at 07:07
  • I have a repro here : https://github.com/spring-cloud/spring-cloud-function/issues/608#issuecomment-816510656 . The key is that on Hoxton.SR10 deserialization of the payload is slightly different when consuming Message, and depending on the objectMapper configuration you get different results. IMHO deserialization of the payload should be the same in both cases. Apparently this issue has been solved in Cloud 2020.0.2 – Sébastien Nussbaumer Apr 09 '21 at 08:24
  • 1
    answer accepted : it should work, and will work with Hoxton.SR11 thanks to https://github.com/spring-cloud/spring-cloud-function/commit/8b9051dfebc4be8ba1eb265d639a838a8c645c05 – Sébastien Nussbaumer Apr 09 '21 at 15:36