0

We have an NServiceBus endpoint that currently listens for ProductChanged events and communicates a number of these changes to a third party.

Due to the volume of events we need to process we have scaled to multiple nodes with multiple logical processors each, but have started running into an issue where multiple messages referring to the same product are being processed simultaneously and causing conflicts in the third party system.

Up until now we've only made use of the basic messaging features of NServiceBus, what features doesn't have that would allow us to "lock" a particular product ID so that any incoming messages for a product that's currently being processed with either be discarded or queued for a later retry?

Reading https://docs.particular.net/nservicebus/sagas/concurrency it seems that Sagas might be able to do this, but I'm not 100% sure on this and in any case it feels like it might be overkill for this scenario?

We're using SQL transport, NSB v6

Just to clarify the problem: while we're able to process an arbitrary number of messages simultaneously, we must not process two messages with the same product ID simultaneously.

Richiban
  • 5,569
  • 3
  • 30
  • 42
  • Normally, a given message would be processed by a single concurrent consumer. So something is off if the same event with the same product ID is handled more than once. It indicates at a problem with how the setup you’ve got is working. What transport are you using. How do you scale out? What your topology looks like. Saga might not be even needed here. – Sean Feldman Apr 21 '22 at 14:16
  • I don't mean to say that literally the same message is handled more than once; I mean that there are lots of processes that can modify products and send ProductChanged events (so there may well be two legitimate messages on the queue that have the same product ID) – Richiban Apr 21 '22 at 14:29
  • Then I don't quite understand the problem you're trying to solve. Is the issue that your messages have to have an affinity to the physical endpoint instance processing all of those? Or out of order processing? Or the third party constraints? I feel like you have a requirement from the 3rd party to process messages associated with the same product ID in a specific order or serially, and the scale out has exacerbated it. – Sean Feldman Apr 21 '22 at 14:53
  • "you have a requirement from the 3rd party to process messages associated with the same product ID in a specific order or serially" That's pretty close; it's more specifically that two messages with the same product ID must not be processed in parallel. Other than that the order of messages doesn't matter. The processing of one of these messages takes (on average) 5 - 10 seconds and if two messages with the same product ID are processed at the same time there's a high chance of a conflict occurring (either with our DB or the 3rd party) – Richiban Apr 22 '22 at 10:27
  • answered the questions. – Sean Feldman Apr 22 '22 at 14:08

1 Answers1

1

When messages need to be processed in order they were sent and one at a time, assuming messages are logically grouped, Message Sessions is the feature Service Bus provides to accomplish the job. The receiver can still scale out and handle multiple sessions concurrently but within any given session only one message will be handled at a time.

Unfortunately, NServiceBus does not support sessions. Sagas soul allow you to accomplish the job but would really clutter the solution because you’d have to store all the incoming messages in the saga’s data (state) and rely on timeouts to execute those. Which, in its own way is not that bad if you have a small number of messages. But if the number of messages is large or their payloads are large, saga becomes a pain point rather than a solution.

If you’d like to have message sessions support in the Service Bus transport, you could upvote the feature request here.

Sean Feldman
  • 23,443
  • 7
  • 55
  • 80