1

Let's say there's a service called "CreateOrder". Let's say as well that the service depends on "Inventory" service to check whether the items in the order are available, and also "Location" service that determines whether the shipping destination is within the supported area (not necessarily an efficient design but put that aside for now).

In the classical approach it is easy. The client just call the "CreateOrder" service, then CreateOrder will call Inventory and Location services (either sequentially or parallel). Only then when both services returns "OK" that CreateOrder return successfully to the client with an OrderId, for example.

But how about the event driven approach? In my understanding an event driven approach works like this (please correct me if I'm wrong):

  1. The client calls the CreateOrder service
  2. Create order post the request into a message broker (let's say kafka) into a pre-defined topic (let's say "Inventory" and "Location" topics)
  3. The Inventory and Location service consume the event from the message broker and process the request

But then, what? How the CreateOrder service get the result so it can get back to the client? One approach that come to mind is that the Inventory and Location services can write to somewhere upon completion (maybe a database), which the CreateOrder service can regularly poll for a result based on some kind of identifier (presumably a request ID). But this sounds horribly inefficient to me.

Is there a better approach? Another idea that come to mind is that the Inventory and Location service can push an event to kafka again upon completion, which the CreateOrder service can subscribe to. But there are a lot of questions with this approach: there are pontentially many requests that are running in parallel, how can the CreateOrder service selectively get the event for the appropriate request? To make it more complicated, what if the "CreateOrder" service is deployed in multiple instances in multiple machine? If this is even possible, is it efficient to do so? And is this possible at all? How to make the RestController wait for the kafka event in the first place?

Please enlighten me. A sample code (preferably in Spring-Boot) is always welcome!

Thanks.

Mycotina
  • 369
  • 3
  • 11
  • 1
    Welcome to eventual consistency. Yes, the client can poll the status of their request. That's a very common solution. Don't couple Kafka processes to an HTTP Response other than 201 Accepted (async request), or 40x, 50x error codes – OneCricketeer Apr 21 '23 at 13:41
  • 1
    _How to make the RestController wait for the kafka event in the first place_ - that's the purpose of Spring Kafka ReplyingKafkaTemplate class – OneCricketeer Apr 21 '23 at 13:44
  • Please see my comment on Levi Ramsey answer, I wonder if you could give feedback on that. For the meantime, I'll check for the ReplyingKafkaTemplate – Mycotina Apr 22 '23 at 04:25

2 Answers2

1

One nice benefit of asynchronous event-driven systems is that they make the latency inherent in disseminating information that is fundamental to one's real problem apparent. Since this latency is a fundamental part of the real world (the speed of light exists, for starters), this means that there's an easy way to think of a solution: figure out how you'd solve it with people communicating and you can then transfer that solution to code and basically treat computers running software as people who can process paperwork really quickly and communicate somewhat reliably.

So imagine you're what used to be called a mail-order or direct-sales merchant. People call you up saying what they'd like to buy.

What would the process for such a phone call be? Are you going to put the caller on hold (better have some really good on-hold music...) while you call the warehouse to validate that what they want is actually in stock?

Chances are, you'll write the order down without putting the caller on hold. You give them an order number that they can use to inquire about the order, and you've probably also obtained some means by which you can contact them if there's some issue with the order. Once you've obtained that information, you say "thank you, your estimated ship time is... and you should get them delivered by..." and you can hang up. Then you forward the order to the warehouse for fulfillment.

Note that, unless you keep the caller on the phone until the package is delivered, you're going to have to provide them with the order number and/or be able to contact them later if there's some problem (that's a pretty boring call, though you're kind of like a sportscaster, which is fun, at least the first time: "and the warehouse worker has your item, and they tripped..."). The data returned by the inventory service is out of date by the time the order creation service receives it.

So your order creation service considers the order created when it's been durably published (to Kafka in this case). 201 Accepted is, as OneCricketeer notes, a perfect HTTP code for that. The other services will publish notable events about the state of the order (e.g. out of stocks, ready for shipment, payment reserved, etc.) for the consumption of a service which allows the requestor (or their designee) to query for a view (hopefully reasonably current) of the order. There might even be services watching those events in order to send proactive communications.

Of course, the obvious question is what if you want to be able to alert the requestor that some items are out of stock. Keeping in mind that in any request-response interaction, by the time the requestor receives the response it's stale (in our phone example, the warehouse could say that they have six widgets, hang up, and then a forklift goes haywire and drives into a shelving unit... oopsie), this implies that the process of order creation is, well, a process. Creating an order isn't a one-shot request-response, it's a dialogue, each request working an in-progress order that can then be finalized. This is where CQRS can come in handy: your order creation service maintains a view of enough of the inventory state to fulfill its duties. In our phone example, this could be a post-it note on the desk with notable out-of-stocks from the warehouse.

Levi Ramsey
  • 18,884
  • 1
  • 16
  • 30
  • Alright, so you're suggesting that the CreateOrder service returns immediately with an OrderID after it feed an event to kafka, which later can be inquiried. But my concern with this approach: How often that the client should check for the order status? Too fast can potentially burden the system (e.g the database), too slow means bad user experience, or even worse, if CreateOrder is actually a dependency of some other services, it means slowing down the system in general. – Mycotina Apr 22 '23 at 04:22
  • The client should be able to be the one to decide whether it needs to and how often to check the order status. Why should you care? If that query load is problematic, CQRS again comes in handy: you can generally scale the read model (the portion of the system needed for queries) to an arbitrary degree and take advantage of things like in-memory caching and pre-computing expensive frequent queries. I'm not sure what you mean by "if CreateOrder is a dependency of some other services, it means slowing down the system in general". – Levi Ramsey Apr 22 '23 at 11:29
  • Why should you care? -- Because I believe that the performance is not the only metrics we are dealing with. While there's no theoretical limitation how much we can scale given a good design, budget is always limited. Some service providers charge you for every request being made. So I'm trying to assess my alternatives and trade offs. – Mycotina Apr 22 '23 at 12:02
  • 1
    @Mycotina - Back to the people modeling example. You're at a restaurant. How often do you go ask your waiter or chef where the food you'd ordered is? The waiter might notify you if it's taking longer than expected, and you have some expectations about how long it should take, but you're certainly not doing it multiple times a second, as you could do with a machine. So, why are you needing to hit a database more frequently than every few seconds or minutes? – OneCricketeer Apr 22 '23 at 13:02
  • @OneCricketeer I think you've a point. In most system I think what you've described is indeed sufficient. But then, I was thinking about the real time chatting application. There's no telling when the next message will arrive. It can be in the next second, but it can also be the next day. I wonder what is our options in such? – Mycotina Apr 23 '23 at 03:03
  • 1
    @Mycotina Those systems often use bidirectional protocols like websockets and gRPC not unidirectional message queues. But even so, linkedin, Facebook chat, etc all send emails when you don't immediately read a message. So, as this answer says - there's a fallback method of contacting the original sender of the message – OneCricketeer Apr 24 '23 at 14:36
0

The Inventory and Location services can also have their own topics for the purpose of streaming/logging their responses. Your order service can subscribe to these topics and filter on keys (i.e. OrderId) that they care about. Since your order thread is waiting for a pertinent event from both topics, you will need to consider timeout scenarios/error handling.

firstblud
  • 135
  • 3
  • 11
  • This looks interesting, but I'm struggling to find the correct API. A sample/documentation links would be appreciated – Mycotina Apr 22 '23 at 05:23
  • @Mycotina it's just Kafka topics. No "api" beyond producer and consumer. I.e have an `order-created` topic. The "factory" (consumer) "sees this order", creates output to `order-started` topic, eventually things happen to get to `order-fulfilled`, and something like an email/sms client for example could be used to notify the orderer that the order is ready – OneCricketeer Apr 22 '23 at 13:07