0

I have been using Azure Service Bus Topics in a production application. My understanding is the system guarantees at least one copy of the message on the subscription end when published.

In practice, I generally see 1 message corresponding to a publish. In some degenerate cases though I see multiples. For example: 1 publish that corresponds to ~20 messages in the relevant subscription. Currently nothing is consuming this subscription, which leads me to believe this is the publishing logic.

Under what circumstances are messages duplicated rampantly? I would imagine networking disturbances might cause this. What else?

Note that the default duplicate handling logic can't handle this. You would have multiple messages with the same body and distinct message IDs - which I understand is used to establish message "identity".

EDIT: My publishing logic is pretty uncomplicated and resembles the following:

public void PublishSomeId(Guid someId, int eventDelayInMinutes, string componentName)
        {
            var topicClient = ServiceBusManager.GetTopicClient();

            var message = new BrokeredMessage(someId);
            message.To = componentName;
            message.ScheduledEnqueueTimeUtc = DateTime.UtcNow.AddMinutes(eventDelayInMinutes);
            topicClient.Send(message);
        }
Arxo Clay
  • 876
  • 6
  • 12
  • More details would be required to answer this question. What's your publishing logic? If you publish the same payload 20 times with a different ID, then yes, you'll have 20 "copies" of the event. What your topology looks like? Without this information answering this is very much guessing. – Sean Feldman Jan 24 '17 at 22:35
  • I have edited the question to add a code sample that closely resembles my actual implementation. Anecdotally I see ~20 messages with body 'someId'. With the help of logs, I know this method has only been invoked once with 'someId'. – Arxo Clay Jan 24 '17 at 22:45
  • Indeed, nothing fancy. What's your setup of topic and subsides under that topic? – Sean Feldman Jan 25 '17 at 00:34
  • I have a topic with 10 subscriptions under it. – Arxo Clay Jan 25 '17 at 00:54
  • I get that. Something in your setup is different though. Sender is dead simple. So is receiver. Topology is missing. Need to see it. Ping me at my email lastname.firstname at gmail. – Sean Feldman Jan 25 '17 at 01:00

1 Answers1

1

Two suggestions:

  1. Verify that your publisher is not duplicating the messages. Add a log statement in PublishSomeId to see what's the someId value is. I suspect this method could be invoked multiple times. In addition to that, if you don't see duplicates logged, you could enable duplicates detection. For that, you should assign someId to the brokered message MessageId. Not sure what's the period of time that duplicates are generated, but you could experiment.

  2. If #1 is not helping, it could the the topology setup (entities). Check there's nothing funky like fun out with fan in back into the same topic.

Sean Feldman
  • 23,443
  • 7
  • 55
  • 80
  • I have marked this answer as accepted as it provides some insight into the next steps I can take to track down this problem. Additional answers explaining what reasons cause duplication to occur are appreciated. – Arxo Clay Jan 25 '17 at 02:08
  • @ArxoClay did you find the source of the duplication? – Sean Feldman Jan 31 '17 at 08:09
  • 1
    We found one source of duplication. I suspect there might be more out there. The one source we found works like this: If you're updating the Subscription while still handling messages on the subscription it seems to affect the locks on messages currently being processed off of that subscription. We have this pattern of "putting back" the message on the subscription (polling scenarios), so in our case the original case can't be 'completed' and a duplicate is placed on the subscription by our code. – Arxo Clay Feb 01 '17 at 17:26