Azure Function : how to implement delay of retrying queue message better

Question

My Azure function should listen Queue for messages, then get message, try to call an external service with value inside message, if the external service returns "OK" then we have to write message to another queue (for next Azure Function), if returns "Fail" we have to return to our current queue and retry by our Azure Function again after 5 minutes. How to implement it? I did it with Timer, but solution does not like me:

    [FunctionName("FunctionOffice365VerificateDomain_and_AddService_and_GexMxRecord")]
    public async static Task Run([TimerTrigger("0 */5 * * * *")]TimerInfo myTimer,
        [Queue("domain-verificate-Office365-add-services-get-mx-record", Connection = "StorageConnectionString")]CloudQueue listenQueue,
        [Queue("domain-add-mx-record-to-registrator", Connection = "StorageConnectionString")]CloudQueue outputQueue,
        ILogger log)
    {
        while (true)
        {
            // do "invisible" message for next 30 sec
            var message = await listenQueue.GetMessageAsync();
            if (message != null)
            {
                DomainForRegistration domainForRegistration = JsonConvert.DeserializeObject<DomainForRegistration>(message.AsString);
                try
                {
                    await _office365DomainService.VerifyDomainAsync(domainForRegistration.DomainName);
                    // remove message
                    await listenQueue.DeleteMessageAsync(message);

                    await _office365DomainService.UpdateIndicateSupportedServicesDomainAsync(domainForRegistration.DomainName);

                    var mxRecord = await _office365DomainService.GetMxRecordForDomainAsync(domainForRegistration.DomainName);
                }
                catch (DomainVerificationRecordNotFoundException)
                {
                     // thrown when VerifyDomainAsync failed
                }
            }
            else
                break;
        }
    }

How to do it more carefully, without these while(true), but with timeout after fail validation?

If this is a queue, why doesn't the function get triggered for every single message? Not sure why you need to loop. — DavidG, Jan 21 '19 at 02:39
@DavidG because after fail I have to add the same message and the function will be called immediately (without delay) again — Oleg Sh, Jan 21 '19 at 14:45

score 3 · Accepted Answer · answered Jan 21 '19 at 09:51

Agree with @DavidG, try to use queue trigger to achieve your goal. W can rely on the host setting of Queue.

visibilityTimeout is The time interval between retries when processing of a message fails maxDequeueCount is The number of times to try processing a message before moving it to the poison queue.

{
    "version": "2.0",
    "extensions": {
        "queues": {
            "visibilityTimeout" : "00:05:00",
            "maxDequeueCount": 2,
        }
    }
}

In this way, the function should look like

public static async Task Run(
    [QueueTrigger("domain-verificate-Office365-add-services-get-mx-record")]string myQueueItem, ILogger log,
    [Queue("domain-add-mx-record-to-registrator", Connection = "StorageConnectionString")]IAsyncCollector<string> outputQueue
)
{
    // do stuff then output message
    await outputQueue.AddAsync(myQueueItem);
}

If you don't want to throw the exception to host, we can turn to initialVisibilityDelay of CloudQueue method.

specifying the interval of time from now during which the message will be invisible

    public static async Task Run(
        [QueueTrigger("domain-verificate-Office365-add-services-get-mx-record")]string myQueueItem, ILogger log,
        [Queue("domain-add-mx-record-to-registrator", Connection = "StorageConnectionString")]IAsyncCollector<string> outputQueue,
        [Queue("domain-verificate-Office365-add-services-get-mx-record", Connection = "StorageConnectionString")]CloudQueue listenQueue
    )
    {

        try 
        {
            // do stuff then output message
            await outputQueue.AddAsync(myQueueItem);
        }
        catch(DomainVerificationRecordNotFoundException)
        {
            // add the message in current queue and can only be visible after 5 minutes
            await listenQueue.AddMessageAsync(new CloudQueueMessage(myQueueItem), null, TimeSpan.FromMinutes(5), null, null);
        }
    }

Thank you! I did not know about "delay" parameter for adding message to queue :) — Oleg Sh, Jan 21 '19 at 15:44

Azure Function : how to implement delay of retrying queue message better

1 Answers1