2

I have a handler whose job it is to take a list of objects, and for each item in that list of objects, Publish an event. That may look something like the code below:

Handle(PublishListMessage message) {
    foreach(var entry in message.List) {
        Bus.Publish(entry);
    }
}

My use case is that I expect message.List.Count to be great, and the foreach loop to potentially therefore take some time.

Therefore, suppose I have processed 50 out of 100 entry objects, then Bus.Publish(entry) fails for some reason. The Handler will retry as per my retry policy, however, it will now begin processing all 100 entries from the beginning.

This is not ideal, so I wanted to persist progress somewhere. Since I am using MongoDB as a persistence layer, I thought I could wrap my Handler in a Saga. The Saga will keep track of all of the entries that have been processed, the hope being that if the Handler fails, it will retry and it will retrieve the Saga taking note of how much progress it has previously achieved.

However, my quick tests have led me to assume that a Saga is not committed (in this case to MongoDB) until the Handler has completed executing. So for my use case, this is not helpful.

My main question is if I can commit a Saga to the DB at some point before the Handler has run to completion. This gives me some of the other benefits of Saga without me having to write my own persistence after every entry has been sent off for Publishing.

My second question is, if this is indeed possible, should I do it in this particular example? Is this a valid use of Saga or is an alternative approach appropriate?

mike
  • 1,318
  • 3
  • 21
  • 41

1 Answers1

1

Why do you expect bus.Publish to fail? This can only happen if your tranport goes down but in this case NServiceBus will set the circuit breaker to armed state and the whole endpoint will be shutdown very shortly. In such case I would be prepared for the whole infrastructure to be down and not reply on some db to hold your progress status. I do not see how saga would help you. If you really do not trust NServiceBus as your infrastructural component on a simple publish, you can save the loop state yourself inside the simple message handler, using the same identifier as you would use for a saga. This could be, for example, the correlation id. Nothing stops you from doind database operations within the message handler, you need no saga for this.

Alexey Zimarev
  • 17,944
  • 2
  • 55
  • 83
  • I think my struggle comes partially from a lack of understanding of the transaction scope of a handler. I have the default ambient transactions set with RabbitMQ as my transport layer. Suppose the VM running RabbitMQ goes down now, while I'm halfway through publishing. What happens? Another case for me is actually that some validation occurs on the instances in the collection prior to publishing. They should all run to completion, but I cannot guarantee that all validations ever will run without throwing any exception. – mike Apr 27 '15 at 20:28
  • I would skip the transport failure handling and leave this to the SRL of your looping handler, the chance of it is too low. Validation should happen in the resulting message handler, not in the loop. Then, validation failure will fail one specific message. There is no need for throwing and retries there. If you choose to validate in the loop, it does not need to fail the handler, just not sent one message. – Alexey Zimarev Apr 28 '15 at 10:25