0

It is often said that Kafka works well with domain driven designs.

Why is it then that Kafka blog posts mostly talk about CQRS or similar - suggesting seperate input and output topics?

It seems like a topic could be about a thing. Why should services 'talk' about that same thing spread out by an implementation detail of who/what is talking?

Isn't this a lot of overhead to protect services from peers that have issues causing them to spam the topic?

I'm hoping for responses that offer pros/cons - why people might think a given thing. Not opinions about the 'right' answer. If this is a better fit for a different SO, I'd appreciate being pointed the right direction.

Levi Ramsey
  • 18,884
  • 1
  • 16
  • 30
russellpierce
  • 4,583
  • 2
  • 32
  • 44
  • Hi, can you, please, add what kind of problem you are trying to solve with Kafka ? – themoah Nov 09 '22 at 11:05
  • Hi Russell. An event store is required for CQRS, it could be a DB, or one of many solutions. Kafka could be used as one of them (but probably it's not the best solution for this). I don't understand the point about input/output topics. There's nothing in CQRS about this, only that the model for processing and querying can be different. One can implement CQRS without using Event Sourcing (they are 2 orthogonal things, but they can be used together). Also, it's not clear what you mean with 'spamming a topic'. – Augusto Nov 09 '22 at 11:10
  • 1
    The question could perhaps apply to any event or message stream. – russellpierce Nov 09 '22 at 12:32
  • 1
    Consider, e.g. https://www.hivemq.com/blog/mqtt5-essentials-part9-request-response-pattern/. Two topics are recommended. Same here: https://www.kai-waehner.de/blog/2022/06/03/apache-kafka-request-response-vs-cqrs-event-sourcing/. – russellpierce Nov 09 '22 at 12:35
  • The aim is to maintain all information about a given domain in a single topic for ease of querying the resulting data. So that if a service flags a domain member as needing a certain update (a request) another service might handle that request with an update to the domain object (a response). – russellpierce Nov 09 '22 at 12:39
  • Re: spamming the topic. I just meant that reuse of the same topic leaves all consumers of the topic vulnerable to a producer that sends too many messages, forms of positive feedback (message amplification where a single input message results in more than one output...), or cross service infinite loops. – russellpierce Nov 09 '22 at 12:40
  • What exactly do you mean by separate input and output topics? Every topic (to a first approximation) is both an input topic and an output topic, unless we're viewing that from the perspective of a single service, though that's orthogonal to a single-writer-type principle that seems to be assumed elsewhere in the question. – Levi Ramsey Nov 10 '22 at 02:42
  • It should also be noted that (at least in Kafka), multiple topics is basically zero overhead (especially relative to the alternative of more partitions). – Levi Ramsey Nov 10 '22 at 02:43
  • @LeviRamsey I mean I see people recommending a 'request' topic as seperate from a 'response' topic. The overhead in more topics is that you lose order guarantees and that someone has to mush the streams back together. – russellpierce Nov 11 '22 at 15:46

1 Answers1

1

To summarize

CQRS is a pattern you use within a service (Bounded context), that allows you to implement the command and query sides in separate ways. That is what CQRS is all about.

Event sourcing is an optional pattern that you often see together with CQRS, but it is not a requirement.

I see Kafka as the backbone between services, as the record of truth between services, like how the picture below:

enter image description here

In this case, you use Kafka to pass notifications of what happens in the different services. (Event-driven architecture). But I would not try to use Kafka for request/response communication even though it is possible.

When you aim for a request/response pattern, you typically want a synchronous response, like if the user sends a command to the system. But in general, to reduce coupling you should not send commands between services, better to publish events and let other services react to those events.

Tore Nestenius
  • 16,431
  • 5
  • 30
  • 40