2

We have several services that publishes and subscribes to Domain Events. What we usually do is log events whenever we publish and log events whenever we process events. We basically use this to apply choreography pattern.

We are not doing Event Sourcing in these systems, and there's no programmatic use for them after publishing/processing. That's the main driver we opted not to store these in a durable container, like a database or event store.

Question is, are we missing some fundamental thing by doing this? Is storing Events a must?

froi
  • 7,268
  • 5
  • 40
  • 78
  • If you are fulfilling all the requirements asked from you, why do you bother yourself? – cool Aug 14 '20 at 14:23

3 Answers3

1

I consider queued messages as system messages, even if they represent some domain event in an event-driven architecture (pub/sub messaging).

There is absolutely no hard-and-fast rule about their storage. If you would like to keep them around you could have your messaging mechanism forward them to some auditing endpoint for storage and then remove them after some time (if necessary).

You are not missing anything fundamental by not storing them.

Eben Roux
  • 12,983
  • 2
  • 27
  • 48
0

You're definitely not missing out on anything (but there is a catch) especially if that's not a need by the business. An Event-Sourced System would definitely store all the events generated by the system into a database (or any other event-store)

The main use of an event store is to be able to restore the state of the system to the current state in case of a failure by replaying messages. To make this process of recovery faster we have snapshots.

In your case since these events are just are only relevant until the process is completed, it would not make sense to store them until you have a failure. (this is the catch) especially in a Distributed Transaction case scenario.

What I would suggest?

  • Don't store the event themselves but log the relevant details about these events and maybe use an ELK stack or Grafana to store these logs.
  • Use either the Saga Pattern or the Routing Slip pattern in case of a Distributed Transaction and log them as well.
  • In case a failure occurs while processing an event, put that event into an exception queue and handle it. If it's a part of a distributed transaction make sure either they all have the same TransactionId or they have a CorrelationId so you can lookup for logs and save your system.
iam.Carrot
  • 4,976
  • 2
  • 24
  • 71
0

For reliably performing your business transactions in a distributed archicture you somehow need to make sure that your events are published at least once.

So a service that publishes events needs to persist such an event within the same transaction that causes it to get created.

Considering you are publishing an event via infrastructure services (e.g. a messaging service) you can not rely on it being available all the time.

Also, your own service instance could go down after persisting your newly created or changed aggregate but before it had the chance to publish the event via, for instance, a messaging service.

Question is, are we missing some fundamental thing by doing this? Is storing Events a must?

It doesn't matter that you are not doing event sourcing. Unless it is okay from the business perspective to sometimes lose an event forever you need to temporarily persist your event with your local transaction until it got published.

You can look into the Transactional Outbox Pattern to achieve reliable event publishing.

Note: Logging/tracking your events somehow for monitoring or later analyzing/reporting purpose is a different thing and has another motivation.

Andreas Hütter
  • 3,288
  • 1
  • 11
  • 19