Abstracting Spring Cloud Stream Producer and Consumer code

Question

I have a Service that is Producing and Consuming messages from different Spring Cloud Stream Channels (bound to EventHub/Kafka topics). There are several such Services which are setup similarly.

The configuration looks like below

 public interface MessageStreams {
      String WORKSPACE = "workspace";
      String UPLOADNOTIFICATION = "uploadnotification";
      String BLOBNOTIFICATION = "blobnotification";
      String INGESTIONSTATUS = "ingestionstatusproducer";

      @Input(WORKSPACE)
      SubscribableChannel workspaceChannel();

      @Output(UPLOADNOTIFICATION)
      MessageChannel uploadNotificationChannel();

      @Input(BLOBNOTIFICATION)
      SubscribableChannel blobNotificationChannel();

      @Output(INGESTIONSTATUS)
      MessageChannel ingestionStatusChannel();
    }


    @EnableBinding(MessageStreams.class)
    public class EventHubStreamsConfiguration {
    }

The Producer/Publisher code looks like below

    @Service
    @Slf4j
    public class IngestionStatusEventPublisher {
      private final MessageStreams messageStreams;

      public IngestionStatusEventPublisher(MessageStreams messageStreams) {
        this.messageStreams = messageStreams;
      }

      public void sendIngestionStatusEvent() {
        log.info("Sending ingestion status event");
        System.out.println("Sending ingestion status event");
        MessageChannel messageChannel = messageStreams.ingestionStatusChannel();
        boolean messageSent = messageChannel.send(MessageBuilder
            .withPayload(IngestionStatusMessage.builder()
                .correlationId("some-correlation-id")
                .status("done")
                .source("some-source")
                .eventTime(OffsetDateTime.now())
                .build())
            .setHeader("tenant-id", "some-tenant")
            .build());
        log.info("Ingestion status event sent successfully {}", messageSent);
      }
    }

Similarly I have multiple other Publishers which publish to different Event Hubs/Topics. Notice that there is a tenant-id header being set for each published message. This is something specific to my multi-tenant application to track the tenant context. Also notice that I am getting the channel to be published to while sending the message.

My Consumer code looks like below

    @Component
    @Slf4j
    public class IngestionStatusEventHandler {
      private AtomicInteger eventCount = new AtomicInteger();

      @StreamListener(TestMessageStreams.INGESTIONSTATUS)
      public void handleEvent(@Payload IngestionStatusMessage message, @Header(name = "tenant-id") String tenantId) throws Exception {
        log.info("New ingestion status event received: {} in Consumer: {}", message, Thread.currentThread().getName());

        // set the tenant context as thread local from the header.

      }

Again I have several such consumers and also there is a tenant context that is set in each consumer based on the incoming tenant-id header that is sent by the Publisher.

My questions is

How do I get rid of the boiler plate code of setting the tenant-id header in Publisher and setting the tenant context in the Consumer by abstracting it into a library which could be included in all the different Services that I have.

Also, is there a way of dynamically identifying the Channel based on the Type of the Message being published. for ex IngestionStatusMessage.class in the given scenario

I may be reading it wrong, but isn't that just two lines of "boilerplate"? As for the channel/class connection, have you considered an enum to keep channel, name and class together? — daniu, Apr 20 '20 at 07:38
A related question: https://stackoverflow.com/questions/61775840/spring-cloud-stream-3-0-1-adding-channel-interceptor-around-input-annotation — Artem Bilan, May 13 '20 at 14:12

score 1 · Accepted Answer · answered Apr 20 '20 at 13:31

1

To set and tenant-id header in the common code and to avoid its copy/pasting in every microservice you can use a ChannelInterceptor and make it as global one with a @GlobalChannelInterceptor and its patterns option.

See more info in Spring Integration: https://docs.spring.io/spring-integration/docs/5.3.0.BUILD-SNAPSHOT/reference/html/core.html#channel-interceptors

https://docs.spring.io/spring-integration/docs/5.3.0.BUILD-SNAPSHOT/reference/html/overview.html#configuration-enable-integration

You can't make a channel selection by the payload type because the payload type is really determined from the @StreamListener method signature.

You can try to have a general @Router with a Message<?> expectation and then return a particular channel name to route according that request message context.

See https://docs.spring.io/spring-integration/docs/5.3.0.BUILD-SNAPSHOT/reference/html/message-routing.html#messaging-routing-chapter

answered Apr 20 '20 at 13:31

Artem Bilan

113,505
11
91
118

The ChannelInterceptor doesnt seem to be available for SubscribableChannel. Is there any way of handling this for Subscribable Channels – java_geek May 12 '20 at 08:17
I have actually tried a different way of handling this; I have an abstract base class called EventHandler from which all of my different EventHandlers extend. I tried to invoke the preProcess and postProcess methods in my Base class whenever handleEvent is called through AOP. However, it seems like the Handler is actually invoked through reflection and is not actually a proxy due to which the advice is not getting applied. – java_geek May 12 '20 at 08:22
public abstract class EventHandler { protected EventHandler() { } public void preProcess(Message message) { // extract headers and do some preprocessing } protected abstract void handleEvent(Message message) throws Exception; public void postProcess(Message message) { //do some post processing } } – java_geek May 12 '20 at 08:24
Cast your channel to `InterceptableChannel` for adding `ChannelInterceptor` – Artem Bilan May 12 '20 at 15:01
I tried casting my channel to InterceptableChannel and added an interceptor during ApplicationContext initialization; however the Interceptor was not invoked – java_geek May 13 '20 at 10:22

Abstracting Spring Cloud Stream Producer and Consumer code

1 Answers1