0

Part of the application I am working on involves the download of emails containing attachments, detecting a barcode in each attachment, scanning it for an identifier and using the identifier to insert data into a database.

So far my usage of message buses has been limited to Pub/Sub and I've never really had the need to use Sagas. However, given that each of the above steps is part of the same related process, and we want durability should any part of the process fail, is a Saga a good fit?

Details of our process and how I thought we could use a Saga are as follows:

  1. A Quartz job is used to download emails that contain PDF attachments. The Quartz job uses a Connector that includes information about the Tenant. This tenant information will be needed by the Saga so we can interact with tenant specific resources (database/storage).
  2. For each PDF attachment we kick off a "Process Document" Saga passing it the tenant information and the document itself (assuming the Message Bus can persist this) or the path of the file on disk. After a Saga has been started for each attachment on the source email, the connector can remove/archive the email from the mailbox so that it is not processed again.

Each Saga will then perform the following tasks:

  1. Attempt to read a barcode from the attachment/file. If the barcode detection fails we would need to upload the file to a specified location so it can be processed manually. This would complete the Saga. If a barcode was detected we decode a database identifier.
  2. Using the identifier we load the relevant Order record from our database and if it exists:
    1. Upload the file to a specified location returning the URI
    2. Insert a new OrderDocument record into our database that references the file URI.
  3. We update a OrderDocumentBundle that is a single file containing all documents linked to an order. So when a new OrderDocument is created we should load the OrderDocumentBundle for the related Order and append the pages from the new OrderDocument.
  4. The Saga completes.

I am concerned about concurrency issues during the final bundling process as we may have multiple messages that attempt to update the bundle. We would need to ensure that the Saga waits for the bundle file to be available so that it complete.

Ben Foster
  • 34,340
  • 40
  • 176
  • 285
  • What's the question? "Is Saga a good choice" is not constructive. – JJJ May 24 '13 at 12:24
  • I've explained our process. I'd like to know if a Saga is the best approach for this. – Ben Foster May 24 '13 at 12:28
  • If you're just using a simple job scheduling, maybe you don't even use Quartz if you opt to use "NServiceBus saga", but just make yourself a favor, when using saga, never publish events from it. – MeTitus May 26 '13 at 23:47

1 Answers1

3

Disclaimer: When talking about Sagas, we're talking about NServiceBus sagas, I hope no religious discussion starts. :)

You should define two or three sagas. One for the email, and another for an attachment.

  • The EmailSaga would know how many attachments there are and not finish until each AttachmentSaga has returned an answer.
  • Each AttachmentSaga can start with calling a handler (RetrieveBarCodeHandler) to retrieve the barcode. If this handler fails, it also replies but with a failure status.
  • The AttachmentSaga can call another saga (ManualProcessingSaga) to store the attachment and email a user (or something like it & it can call other handlers) to manually process the barcode.
    • The user manually processes the attachment and submits the result with a command (RegisterBarcodeCommand)
    • The ManualProcessingSaga receives the command and sends a reply to the originator, the AttachmentSaga.
    • The AttachmentSaga can continue its work and finally sends a reply to the EmailSaga
  • The EmailSaga has all the replies and knows it is done.

Why so many sagas? Because

  1. Handling the email is a long running process
  2. Handling the attachment is a long running process
  3. Manually processing the attachment is a long running process

A long running process is not it, because of time, but because multiple things are happening or it is waiting for answers or a timeout.

Of course remember to include something related to timeouts. If an AttachmentSaga does not answer within a certain amount of time, notify a user or something.

Dennis van der Stelt
  • 2,203
  • 16
  • 22
  • I was about to start a religious discussion when I saw the saga name, but fortunately I saw the "NServiceBus sagas" in time. – MeTitus May 26 '13 at 23:43
  • The way define the process is not the best IMO because you're allowing your sagas to reply back to other sagas, but then again, you're talking about "NServiceBus sagas" which allow you to do just that. – MeTitus May 26 '13 at 23:50
  • After looking into this in more detail I'm not sure I see the benefit of having the parent `Email` over just starting a new Saga for each for attachment. – Ben Foster May 27 '13 at 15:28
  • Not sure if I understand what you mean. I you want, email me. My blog (and contact page) can be found in my profile. – Dennis van der Stelt May 28 '13 at 19:00