Multi channel conditional routing in spring cloud stream

Question

I am using spring cloud stream kafka with functional approach. Have got two topics Topic1 and Topic2 and contains messages related to multiple schema.

Topic1 contains messages of Car and Bike

Topic2 contains messages related to Fruits and Vegetables.

Would like to process them independently as processCar, processBike, processFruits and ProcessVegetables functions.

Able to apply MessageRoutingCallback functionality to handle Car and Bike from one topic. But unable to do so on topic2.

    @Bean
    public MessageRoutingCallback customRouter() {
        String EVENT_TYPE = "eventType";

        Map<String, String> evenTypeToFunctionNameMap = new HashMap<>();
        evenTypeToFunctionNameMap.put(
                "car",
                "processCar"
        );
        evenTypeToFunctionNameMap.put(
                "bike",
                "processBike"
        );
        

        return new MessageRoutingCallback() {
            @Override
            public FunctionRoutingResult routingResult(Message<?> message) {
                return new FunctionRoutingResult(
                        evenTypeToFunctionNameMap.get(
                                message.getHeaders().get(EVENT_TYPE, String.class)
                        )
                );
            }
        };
    }

And properties are

spring.cloud.stream.bindings.functionRouter-in-0.destination=topic1
spring.cloud.stream.bindings.functionRouter-in-0.binder=default

Note : Would like to avoid spring.cloud.stream.bindings.functionRouter-in-0.destination=topic1, topic2 as an option as topic 1 and topic 2 needs to be on different channel as they are on different binder / have different fault processing needs.

score 1 · Answer 1 · answered Jun 14 '23 at 16:32

Solved using multiple router configuration

Custom bean to be defined for each router function,

    @Bean("myRouter")
    RoutingFunction myRouter(FunctionCatalog functionCatalog, FunctionProperties functionProperties, BeanFactory beanFactory, @Nullable MessageRoutingCallback routingCallback) {
        Map<String, String> propertiesMap = new HashMap<>();
                String EVENT_TYPE = "eventType";

        Map<String, String> evenTypeToFunctionNameMap = new HashMap<>();
        evenTypeToFunctionNameMap.put(
                "car",
                "processCar"
        );
        evenTypeToFunctionNameMap.put(
                "bike",
                "processBike"
        );
        

        var cb1 = new MessageRoutingCallback() {
            @Override
            public FunctionRoutingResult routingResult(Message<?> message) {
                return new FunctionRoutingResult(
                        evenTypeToFunctionNameMap.get(
                                message.getHeaders().get(EVENT_TYPE, String.class)
                        )
                );
            }
        };
        return new RoutingFunction(functionCatalog, propertiesMap, new BeanFactoryResolver(beanFactory), cb1);
    }

@Bean("anotherRouter")
    RoutingFunction anotherRouter(FunctionCatalog functionCatalog, FunctionProperties functionProperties, BeanFactory beanFactory, @Nullable MessageRoutingCallback routingCallback) {
        Map<String, String> propertiesMap = new HashMap<>();
                String EVENT_TYPE = "eventType";

        Map<String, String> evenTypeToFunctionNameMap = new HashMap<>();
        evenTypeToFunctionNameMap.put(
                "fruit",
                "processFruit"
        );
        evenTypeToFunctionNameMap.put(
                "vegetable",
                "processVegetable"
        );
        

        var cb1 = new MessageRoutingCallback() {
            @Override
            public FunctionRoutingResult routingResult(Message<?> message) {
                return new FunctionRoutingResult(
                        evenTypeToFunctionNameMap.get(
                                message.getHeaders().get(EVENT_TYPE, String.class)
                        )
                );
            }
        };
        return new RoutingFunction(functionCatalog, propertiesMap, new BeanFactoryResolver(beanFactory), cb1);
    }

@Bean
public Consumer<Car> processCar() {
     return event -> {
     }
}

@Bean
public Consumer<Bike> processBike() {
      return event -> {
      }
}

@Bean
public Consumer<Fruit> processFruit() {
      return event -> {
      }
}

@Bean
public Consumer<Vegetable> processVegetable() {
      return event -> {
      }
}

spring.cloud.stream.function.routing.enabled=true
spring.cloud.function.definition=myRouter;anotherRouter
spring.cloud.stream.bindings.myRouter-in-0.destination=Topic1
spring.cloud.stream.bindings.myRouter-in-0.content-type=application/json
spring.cloud.stream.bindings.anotherRouter-in-0.destination=Topic2
spring.cloud.stream.bindings.anotherRouter-in-0.content-type=application/json

This is pretty good answer. @MusicMan would you consider adding exactly what you have written and few words of explanation to our docs - https://github.com/spring-cloud/spring-cloud-stream/blob/main/docs/src/main/asciidoc/spring-cloud-stream.adoc#routing-function-and-output-binding. Or if you are ok, i'll use your example? I would prefer the first one, this way your name gets in the doc authors — Oleg Zhurakousky, Jun 16 '23 at 06:54
Happy to receive a comment from you @OlegZhurakousky :-) . I will contribute to the samples repo. — MusicMan, Jun 17 '23 at 07:30

Multi channel conditional routing in spring cloud stream

1 Answers1