4

Let's say we have a system were there's one producer that queues messages in a queue and multiple instances of the same consumer processing this events. Since we are in a Competing Consumers pattern we know that ordering is no longerd guaranteed. This means that we must ensure that our messages are idempotent.

From what I read here (under the Message ordering bullet point), we must ensure that message processing is idempotent.

Here's the questions:

  • How can we design our message processing to be idempotent?
  • If we are saving every event in an event store, are there any consideration to be taken into account when designing each event's payload and the events aggregation to get the aggregate state?

An example: let's say that we have a "User Created" and a "User Deleted" messages (or any other couple of events which NEED to be processed in order). If we process "User deleted" before "User created" the user won't be deleted. Even if they're coming ordered in the event queue. Can really an idempotent processing/idempotent events give to a deleted user?

Another example. Let's suppose that we have an entity that have a score attribute. An user can modify the score. A second service consumes events of the "score entity" service and if the score reaches 100 the entity (or an entity reference) is inserted by the second service in the "Best category" entity. If the score reaches -20 the second service insert the score entity in the "Worse category". Having multiple instace of the second service can give an impredictable result if the "score 100" and "score -20" events are within a tiny interval of time. Any ideas on how to design the "score x" events or how to process these events?

Thank you so much for your help!

Christian Paesante
  • 484
  • 6
  • 16

1 Answers1

4

How can we design our message processing to be idempotent?

You should:

  • ignore messages that you have already "seen"; this means that the consumer should have a way of detecting that; it could, for example, keep a list of message IDs that it has processed (this means that every message should have an unique ID).
  • not throw an exception if a message does not change the state; for example, if you receive a second DeleteUser event (so the user is already deleted, a second delete should have no side effect) then you ignore it. Not every event can be idempotent, for example UpdateUserName should not be idempotent.

If we are saving every event in an event store, are there any consideration to be taken into account when designing each event's payload and the events aggregation to get the aggregate state?

You should design the events based on your Domain; the payload should not contain more information than it is needed by the domain. If additional information would make your readmodels a lot easier to implement then you could add it to the payload but be careful to mark it somehow as redundant.

An example: let's say that we have a "User Created" and a "User Deleted" messages (or any other couple of events which NEED to be processed in order). If we process "User deleted" before "User created" the user won't be deleted. Even if they're coming ordered in the event queue. Can really an idempotent processing/idempotent events give to a deleted user?

In this particular case, you can have an additional collection of deleted users; you can keep only their ID. When a CreateUser event arrives you can check to see if the user is already deleted by looking in the DeletedUsers collection and ignore it if the user is there. You can ignore every other event that comes for that user.

This solution is very dependent on the domain.

Another example. Let's suppose that we have an entity that have a score attribute. An user can modify the score. A second service consumes events of the "score entity" service and if the score reaches 100 the entity (or an entity reference) is inserted by the second service in the "Best category" entity. If the score reaches -20 the second service insert the score entity in the "Worse category". Having multiple instace of the second service can give an impredictable result if the "score 100" and "score -20" events are within a tiny interval of time. Any ideas on how to design the "score x" events or how to process these events?

This situation can be resolved by keeping in-the/attached-to readmodels (the two collections, best and worse) the timestamp/order/stream-version/whatever of the last processed event and ignore every event that is less-or-equal to that timestamp. In this way, if "score 100" is emitted after "score -20" but arrives first you should ignore the "score -20" because it has a lower timestamp, although it comes last.

This solution is generic but it relies on the existence of some ordering.

Constantin Galbenu
  • 16,951
  • 3
  • 38
  • 54
  • So, basically all depends on what you are doing. And every solution you implement is different case by case, isn't it? One last question. In a competing consumers pattern, when two events (which hold a timestamp metadata) are emitted one after another but they are received out of order, does it changes anything you said before? From my point of view, unordered processing or unordered receiving (or both) gives to the same situation in a competing consumer pattern, am I right? – Christian Paesante Oct 08 '18 at 16:51
  • @ChristianPaesante in general you need to think and choose the best solution in each use-case. I have observed two cases: it depends on whether the two events are interdependent or independent. For example, one could be caused by the other (directly or indirectly). Independent events should not cause problems when they are received out-of-order. Interdependent events should be treated specially, depending on the use-case. You could ignore newer events that come sooner (like the CreateUser that comes after DeleteUser) or your readmodel could catch-up when an old event comes later than normal – Constantin Galbenu Oct 08 '18 at 17:25
  • @ChristianPaesante In conclusion, I don't think there is a generic solution to this out-of-order problem. – Constantin Galbenu Oct 08 '18 at 17:26