0

Our product consist of several small application, famously known as a Microservice architecture. These services a hosted in Azure Kubernetes Service(AKS) with a HorizontalPodScaler. This means that we dont control how many instances, or Pods, of an application that is running, just the minimum and maximum amount of Pods that the Cluster can scale out.

We're using Azure Service Bus and MassTransit to publish events. As we now want to move from Queues to Topics, we do have some questions that some of you might know the answer to.

Lets say that one of our microservices subscribes to a topic. Kubernetes has found out that its time to scale out to several instances of this particular service. What will happen now? My assumption is that the consumer for this specific topic will get triggered for each running pod of this application. E.G We have three pods of the same microservice, an event is published to the topic, the consumer in all three instances get triggered.

To avoid side-effects of this behavior, I think that each microservice should have its own queue and the topic forwards events to each queue as explained here and here.

Question 1: Is this a good solution and "by design" ?

Question 2: Should each event have its own topic?

How should I configure this in MassTransit? The documentation says that for each queue, I can subscribe to a specific topic.

cfg.ReceiveEndpoint("microservice-1-queue", e =>
{
    e.Subscribe("userdeletedtopic");
    //What about the consumers?
})

Should there be a 1-1 matching between the topics and the queues? Like this pseudocode:

cfg.ReceiveEndpoint("microservice-1-userdeleted-queue", e =>
{
    e.Subscribe("userdeletedtopic");
    e.Consumer<UserDeletedEventConsumer>();
})

Or can we have one queue for each microservice that subscribes to many topics. Like this:

cfg.ReceiveEndpoint("microservice-1-userdeleted-queue", e =>
{
    e.Subscribe("userdeletedtopic");
    //How does MassTransit know which consumer to forward the message to?
    //And if several events are forward to the same queue, how does MassTransit know 
    //how to deserialize the event?
})

Have anyone done the same thing and can pinpoint me or even share some example code for this particular case?

Please feel free to comment if something is unclear.

Thanks in advance.

Tobias Moe Thorstensen
  • 8,861
  • 16
  • 75
  • 143
  • 1
    If you're calling `Publish` in MassTransit, and have already configured receive endpoints, then you're already using topic forwarding to queues. Scaling out, as long as each instance is using the same queue name for the receive endpoint, will scale via competing consumer on that queue. It should just work if you're following the default MassTransit conventions. – Chris Patterson Jun 02 '20 at 16:21
  • Is it really this simple? Wow. Thanks.. – Tobias Moe Thorstensen Jun 03 '20 at 07:00

0 Answers0