0

I'm reasoning about event sourcing and often I arrive at a chicken and egg problem. Would be grateful for some hints on how to reason around this.

If I execute all I/O-bound processing async (ie writing to the event log) then how do I handle, or sometimes even detect, failures?

I'm using Akka Actors so processing is sequential for each event/message. I do not have any database at this time, instead I would persist all the events in an event log and then keep an aggregated state of all the events in a model stored in memory. Queries are all against this model, you can consider it to be a cache.

Example

Creating a new user:

  1. Validate that the user does not exist in model
  2. Persist event to journal
  3. Update model (in memory)

If step 3 breaks I still have persisted my event so I can replay it at a later date. If step 2 breaks I can handle that as well gracefully.

This is fine, but since step 2 is I/O-bound I figured that I should do I/O in a separate actor to free up the first actor for queries:

Updating a user while allowing queries (A0 = Front end/GUI actor, A1 = Processor Actor, A2 = IO-actor, E = event bus).

  1. (A0->E->A1) Event is published to update user 'U1'. Validate that the user 'U1' exists in model
  2. (A1->A2) Persist event to journal (separate actor)
  3. (A0->E->A1->A0) Query for user 'U1' profile
  4. (A2->A1) Event is now persisted continue to update model
  5. (A0->E->A1->A0) Query for user 'U1' profile (now returns fresh data)

This is appealing since queries can be processed while I/O-is churning along at it's own pace.

But now I can cause myself all kinds of problems where I could have two incompatible commands (delete and then update) be persisted to the event log and crash on me when replayed up at a later date, since I do the validation before persisting the event and then update the model.

My aim is to have a simple reasoning around my model (since Actor processes messages sequentially single threaded) but not be waiting for I/O-bound updates when Querying. I get the feeling I'm modeling a database which in itself is might be a problem.

If things are unclear please write a comment.

Magnus
  • 3,691
  • 5
  • 27
  • 35
  • It does not matter, as long as you know that you are not publishing an event without committing your changes to the model or updating the model and not publishing the event. A lot of frameworks provide this kind of transactional context including NServiceBus, using MSDTC, but there are other ways of achieving that. No one can actually give you a concrete answers for that, it really depends on your current infrastructure. – MeTitus Aug 11 '13 at 20:34

1 Answers1

0

Asychronous I/O can coexist with transactional updates. If you send an "ACK" or "NACK" after the command, then you can understand whether it has happened or not. In a distributed or truly asynchronous model, it is likely the "NACK" will come from both explicit failures and time-outs.

Sebastian Good
  • 6,310
  • 2
  • 33
  • 57
  • In a windows system you mean using MSDTC to coordinate that transaction right? In relation to the OP, commands in a true asyn environment end up in a FIFO queue, so I am not sure how you fit the transactions and updating a model in a asyn command context in the same basket. I would understand a transaction grouping the an update in the model and an event being created in an outgoing queue, but not commands. – MeTitus Aug 11 '13 at 20:28