1

I have the following workflow:

  1. Service bus receives messages.
  2. Azure function triggers and tries to deliver this messages via HTTP to some service.
  3. If delivery failed - function throws exception (custom) and disables topic subscription via code below:

enter image description here

  1. The other function in parallel pings special health check endpoint of the service, and if it gets 200 - it tries to enable subscription and make the flow work again.
  2. The steps could be reproduced N times, cause health check will return 200, thus the delivery url of point 2 - 4xx code.

After the next attempt to enable subscription and deliver the message, I expect that delivery count will be increased and in the end (after 10 deliveries attempt) it will get to dead-letter. Actual - it equals 1.

enter image description here

I assume, that it may reset when I call CreateOrUpdate with status changed. If yes - what is the other way to manage subscription status instead of Microsoft.Azure.Management package so that the messages delivery count will not be reset?

UPDATE: Function code

public static class ESBTESTSubscriptionTrigger
{
    private static readonly HttpClient Client = new HttpClient();

    private static IDatabase redisCache;

    [FunctionName("ESBTESTSubscriptionTrigger")]
    [Singleton]
    public static async Task Run([ServiceBusTrigger("Notifications", "ESBTEST", AccessRights.Listen, Connection = "NotificationsBusConnectionString")]BrokeredMessage serviceBusMessage, TraceWriter log, [Inject]IKeyVaultSecretsManager keyVaultSecretsManager)
    {
        var logicAppUrl = await keyVaultSecretsManager.GetSecretAsync("NotificationsLogicAppUrl");

        if (redisCache == null)
        {
            redisCache = RedisCacheConnectionManager.GetRedisCacheConnection(
                keyVaultSecretsManager.GetSecretAsync("RedisCacheConnectionString").GetAwaiter().GetResult());
        }

        if (string.IsNullOrWhiteSpace(logicAppUrl))
        {
            log.Error("Logic App URL should be provided in Application settings of function App.");
            throw new ParameterIsMissingException("Logic App URL should be provided in Application settings of function App.");
        }

        var applicaitonId = serviceBusMessage.Properties["applicationId"].ToString();
        var eventName = serviceBusMessage.Properties.ContainsKey("Event-Name") ? serviceBusMessage.Properties["Event-Name"].ToString() : string.Empty;
        if (string.IsNullOrWhiteSpace(applicaitonId))
        {
            log.Error("ApplicationId should be present in service bus message properties.");
            throw new ParameterIsMissingException("Application id is missing in service bus message.");
        }

        Stream stream = serviceBusMessage.GetBody<Stream>();
        StreamReader reader = new StreamReader(stream);
        string s = reader.ReadToEnd();

        var content = new StringContent(s, Encoding.UTF8, "application/json");
        content.Headers.Add("ApplicationId", applicaitonId);

        HttpResponseMessage response;
        try
        {
            response = await Client.PostAsync(logicAppUrl, content);
        }
        catch (HttpRequestException e)
        {
            log.Error($"Logic App responded with {e.Message}");
            throw new LogicAppBadRequestException($"Logic App responded with {e.Message}", e);
        }

        if (!response.IsSuccessStatusCode)
        {
            log.Error($"Logic App responded with {response.StatusCode}");

            var serviceBusSubscriptionsSwitcherUrl = await keyVaultSecretsManager.GetSecretAsync("ServiceBusTopicSubscriptionSwitcherUri");
            var sbSubscriptionSwitcherResponse = await Client.SendAsync(
                                                     new HttpRequestMessage(HttpMethod.Post, serviceBusSubscriptionsSwitcherUrl)
                                                         {
                                                             Content =
                                                                 new
                                                                     StringContent(
                                                                         $"{{\"Action\":\"Disable\",\"SubscriptionName\":\"{applicaitonId}\"}}",
                                                                         Encoding.UTF8,
                                                                         "application/json")
                                                         });

            if (sbSubscriptionSwitcherResponse.IsSuccessStatusCode == false)
            {
                throw new FunctionNotAvailableException($"ServiceBusTopicSubscriptionSwitcher responded with {sbSubscriptionSwitcherResponse.StatusCode}");
            }

            throw new LogicAppBadRequestException($"Logic App responded with {response.StatusCode}");
        }

        if (!string.IsNullOrWhiteSpace(eventName))
        {
            redisCache.KeyDelete($"{applicaitonId}{eventName}DeliveryErrorEmailSent");
        }
    }
}
BumbleBee
  • 97
  • 1
  • 8
  • 2
    I'm not sure I agree with this design - removing the subscription because a downstream component is down seems drastic. Typically the app would leave messages on the queue if it was unable to process them, and the standard [DLQ](https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-dead-letter-queues) could be used if messages were unprocessed for too long, and compensation could occur. Alternatively, if you didn't care about messages if they can't be delivered, then could set a [shorter TTL](https://docs.microsoft.com/en-us/azure/service-bus-messaging/message-expiration) – StuartLC May 22 '18 at 15:13
  • 1
    @StuartLC hi, not removing the subscription - simply disabling it. Messages are left in this subscription unprocessed, until it's enabled again. And if the subscription was enabled\disabled to much times to delivery count be encountered 10 times (by default) - then the message should go to DLQ. This is the expected behavior. Unfortunately, the counter is not increasing. Thank you for your comment anyway. It's exceptional case for this behavior. In ideal world the system shouldn't trapped in such case. But it's possible, so I'm trying to understand what could be done to fix it. – BumbleBee May 22 '18 at 16:16
  • Can you please post your function code? – Alexey Rodionov May 22 '18 at 18:25
  • @AlexeyRodionov hi, updated with it. – BumbleBee May 22 '18 at 19:57

0 Answers0