Azure ServiceBus Topic with multiple Subscribers - Duplicate messages

Question

I have an Azure ServiceBus Topic called "IntegrationEvent" with one subscription called "TestSubscription". I register two subscribers to this subscription that I have CompetingConsumers.

A lot of messages are handled in both subscribers. What do I need to change that this won't happen anymore? I thought every message should be handled only by one subscriber?

Sender:

class Program
{
    static async Task Main(string[] args)
    {
        await SendMessageAsync();
    }

    private static async Task SendMessageAsync()
    {
        var sendClient = new TopicClient("ConnectionString");

        var testBlock = new ActionBlock<string>(
            async id =>
            {
                string jsonMessage = JsonConvert.SerializeObject(id);
                byte[] body = Encoding.UTF8.GetBytes(jsonMessage);

                var messageToSend = new Message(body)
                {
                    CorrelationId = id,
                };

                await sendClient.SendAsync(messageToSend);

            }, new ExecutionDataflowBlockOptions
            {
                MaxDegreeOfParallelism = 25
            });

        for (int i = 0; i < 10000; i++)
        {
            testBlock.Post(Guid.NewGuid().ToString());
        }

        testBlock.Complete();
        await testBlock.Completion;
    }
}

I use two Subscriber/Consumer (not Subscription) listening to IntegrationEvent.

class Program
{
    static SubscriptionClient subscriptionClient;

    static async Task Main(string[] args)
    {
        var builder = new ServiceBusConnectionStringBuilder("ConnectionString");
        if (string.IsNullOrWhiteSpace(builder.EntityPath))
        {
            builder.EntityPath = "IntegrationEvent";
        }

        subscriptionClient = new SubscriptionClient(builder, "TestSubscription");

        await subscriptionClient.RemoveRuleAsync(RuleDescription.DefaultRuleName);
        await subscriptionClient.AddRuleAsync(new RuleDescription(RuleDescription.DefaultRuleName, new TrueFilter()));

        ListenForMessages();

        Console.Read();
    }

    protected static void ListenForMessages()
    {
        var options = new MessageHandlerOptions(ExceptionReceivedHandler)
        {
            AutoComplete = false,
            MaxConcurrentCalls = 10
        };
        subscriptionClient.RegisterMessageHandler(ReceiveMessageAsync, options);
    }

    private static Task ExceptionReceivedHandler(ExceptionReceivedEventArgs arg)
    {
        return Task.CompletedTask;
    }

    private static async Task ReceiveMessageAsync(Message arg1, CancellationToken arg2)
    {
        string integrationEvent = Encoding.UTF8.GetString(arg1.Body);
        Console.WriteLine($"{ arg1.MessageId}, { arg1.CorrelationId}, {integrationEvent}");
        await subscriptionClient.CompleteAsync(arg1.SystemProperties.LockToken);
    }
}

Answer 1

It's important to understand how subscriptions work before you attempt anything. There's documentation exactly about that. Specifically, rules that affect what the subscriptions are going to receive.

Right now your code is set up to receive all messages (catch all rule). Once you create a more specific rules using SQL or Correlation filters, you'll be able to control what each individual subscription will receive. I wrote a post a while ago that has some of that information.