We are going to leverage the Azure Service Bus Queue to dispatch messages that trigger an expensive database calculation. These messages can be generated fairly rapidly, and there are instances where the same messages can be generated close together. In this case, we only want to process the data once.
So we intend on leveraging Duplicate Detection, but there are some nuances that I cannot glean from the documentation or from previous questions. The parameters are:
- If a message is waiting to be processed, and an identical message is about to be created, DON'T create it.
- If a message was processed previously, and an identical message is about to be created, DO create it.
- If a message is currently being processed, and an identical message is about to be created, DO create it.
What I can't find out is a way to disable "historical" duplicate detection but enable "waiting" duplicate detection - detect messages waiting in the queue but not those that have already been processed. Does the Duplicate detection history mechanism work only on completed messages or is the timeframe also relevant to waiting messages relative to each other?
For #3, I am considering solving that by adding a 5 minute delay to processing any message, so that if one is created while one is processing, there is enough time for the processing one to complete.
