0

I am looking for design guidance here more than actual solutions, but welcome both.

The scenario is that I have a publisher which is responsible for publishing user updates performed in a system. My downstream systems are the subscribers to these updates.

The challenge I am having is that the users of the upstream system are regularly saving their work. The reason they are doing this is because one "logical" update actually involves transitioning many screens plus it is also natural for them to be multitasking on other things and as such they hit "save" a lot. Each time they hit save we receive a message.

So For each "logical" update we have maybe 5-10 individual update messages, some of which could be duplicated.

This causes overhead for the users of my downstream systems, who are being overwhelmed with the volume of updates. For each update they need to first check if this represents an "actionable" piece of work, and discard it if this is only the result of an upstream save.

I know the best solution would be to change the upstream system to batch the saves together into a single update but this would be too costly as this would have to involve the vendor.

Edit: There are no sequencing data in the messages. So when we receive the update it there is no way of telling how "far" through the process the user is. They could have finished it all in one go, or they could only be 10% of the way through and we will get 10 more updates before they are finished.

Edit: specifically I am using BizTalk as the messaging platform.

Edit: The pattern I want to implement is the Aggregator - http://eaipatterns.com/Aggregator.html. The problem is I have no way of knowing when the series of messages which comprise the input is complete or not.

tom redfern
  • 30,562
  • 14
  • 91
  • 126
  • Sorry can't get a main point of the question – sll Dec 15 '11 at 10:38
  • I am looking for suggestions on how others would approach this problem. I need to reduce the volume of messaging to my downstream systems by "batching up" messages in a coherent way – tom redfern Dec 15 '11 at 10:39
  • Perhaps you are looking for the [Message Sequence](http://eaipatterns.com/MessageSequence.html) pattern? So users can start handling the updates when received an update with `SequenceNumber == SequenceSize` – sll Dec 15 '11 at 10:41
  • Thanks for that. However the system which is publishing the messages has no concept of sequence numbers – tom redfern Dec 15 '11 at 10:44
  • Have update my question to try to be a bit more specific. – tom redfern Dec 15 '11 at 10:47
  • Does messages have a `sent timestamp` which is set by a sender so you can leverage this? BTW, many enterprise systems allows configuring connection so you can request additional features like message sequence number, etc, perhaps it worth take a look in the documentation to see whether you can enable such features? – sll Dec 15 '11 at 10:52
  • so perhaps ordering messages by a sent time stamp would be a solution? – sll Dec 15 '11 at 10:57
  • The issue is not so much the order but the volume of messages – tom redfern Dec 15 '11 at 12:54

1 Answers1

0

I decided to implement a variant of the aggregator pattern whereby the default message (the most recent) is chosen from the collective interchange.

However, for the "trigger" (the set of conditions which indicate that the aggregation is complete), rather than rely on the completion of the series I will have a service window set at a certain time each day.

Hope this helps others.

tom redfern
  • 30,562
  • 14
  • 91
  • 126