0

I am currently receiving messages from kafka and the payload is protobuf. A customized MessageConverter is used to decode the content before passing to my function. What I am trying to do now is to inspect the incoming payload and conditionally discard the message if it is not what I want. The issue is that the payload I received in my MessageRoutingCallback::routingResult is raw bytes and my MessageConverter isn't invoked yet. I know that I can manually decode it right in the callback function and dispatch to the corresponding function definition. I am just wondering if there is a better way to wire the things in correct order.

The version of spring cloud function I am using is 3.2.1

Muhammad Dyas Yaskur
  • 6,914
  • 10
  • 48
  • 73
Cecil Wei
  • 3
  • 1

1 Answers1

0

First of all MessageRoutingCallback is for routing. What you are describing is filtering which is quite different from the pure pattern perspective. . .

Regardless, that is actually intentional. I discuss it a little here

IMHO it is absolutely bad idea to pass any non-functional decisions (i.e., routing, filtering, SpEL evaluation etc) based on the payload to the framework, since payload is privileged data. I talk about in in the provided link. Given that everything comes in as Message, you have options to construct your Message in such way that you can make such decisions based on Message headers. . . even if that would require adding some functionality upstream to handle Data Enricher pattern and modify the Message in such way that downstream such non-functional decisions can be performed. Also, consider another use case where routing would send message to either Function<Foo, ?> or Function<Bar, ?>? As you can see from the example, the payload type is different. This means that the routing decision needs to be made before teh actual payload conversion to a concrete type.

Oleg Zhurakousky
  • 5,820
  • 16
  • 17
  • Hi Oleg, I am still struggling with doing message filtering and exception handling with imperative function. I was reading https://codelabs.solace.dev/codelabs/spring-cloud-stream-beyond/#8 and thought that I can simply return null if I want to drop the message. But I saw a null pointer exception is thrown and the message is not consumed (I am using kafka binder). Could you help to advise the proper way of message filtering and error handling (stop the message routing but send it to another consumer)? Thank you very much. – Cecil Wei Dec 23 '21 at 01:29
  • After tracing the source code of spring cloud function (version 3.2.1), I found that returning null value in imperative function only works when there is only one function (the message is dropped in ```org.springframework.cloud.stream.function.FunctionConfiguration$FunctionToDestinationBinder$1.handleMessageInternal```). However, if function composition is used, null return value is not expected and will cause NPE at ```org.springframework.cloud.function.context.catalog.SimpleFunctionRegistry$FunctionInvocationWrapper.convertNonMessageInputIfNecessary(SimpleFunctionRegistry.java:1211)```. – Cecil Wei Dec 23 '21 at 07:07
  • I really like the flexibility of imperative function programming model. I would appreciate any advise on how to properly use it for filtering and error handling in developing with spring cloud stream. – Cecil Wei Dec 23 '21 at 07:15