ConcurrentQueue dequeue problem in Azure function

Question

I have declared a ConcurrentQueue and added a list of GUIDs. Adding into the queue is fine, but when I access the queue from inside the TimerTrigger function it seems like it's empty (updateQueue.count is 0). This behavior is happening in the cloud, but when I execute the same locally it works fine.

public static class Function1
{
    private static readonly ConcurrentQueue<IEnumerable<Guid>> updateQueue = new ConcurrentQueue<IEnumerable<Guid>>();

    public static async Task<IActionResult> UpdateRun(
        [HttpTrigger(AuthorizationLevel.Admin, "post", Route = null)] HttpRequest req, ExecutionContext execContext)
    {
        logger.LogInformation($"FunctionUpdate Start");
        using var reader = new StreamReader(req.Body);
        var request = JsonConvert.DeserializeObject<IEnumerable<Request>>(reader.ReadToEnd());
        var correlationIds = request?.Select(s => s.CorrelationId);

        updateQueue.Enqueue(correlationIds);
        return new OkObjectResult(new Response { HttpStatusCode = HttpStatusCode.Accepted });
    }

    [FunctionName("FunctionHandleQueue"), Timeout("00:05:00")]
    public static async Task HandleQueue([TimerTrigger("0 */1 * * * *")] TimerInfo myTimer, ExecutionContext execContext) // once every 1 minutes
    {
        logger.LogInformation($"before updateQueue condition : {updateQueue.Count}"); 
        if (updateQueue.Count > 0)
        {
            logger.LogInformation($"after updateQueue condition {updateQueue.Count}");

            var guids = new List<Guid>();
            var count = 0;
            while (count <= 1000 && updateQueue.Count > 0)
            {
                updateQueue.TryDequeue(out var updateRequest);
                var enumerable = updateRequest.ToList();
                count += enumerable.Count;
                guids.AddRange(enumerable);
            }

            await new ProcessUpdateSales(CreateMapper(), execContext)
                .Orchestrate(guids)
        }
    }
}

Logs are created when TimerTrigger executes every 1 minute:

before updateQueue condition : 0

Why is updateQueue.Count always 0? What am I doing wrong?

While Azure Functions may *typically* share a single backplane, it is not guaranteed. Resources can be spun down or up at any time and new copies of functions may not have access to the original state. If you need to preserve state, use a durable function or an actual queue, like an Azure Storage Queue. — David L, Jan 15 '21 at 20:55
To add to David's comment: it works locally on your machine because there's just a single local application domain spun up by dev runtime of Functions, so you add and remove from the same queue object. — Roman Polunin, Jan 15 '21 at 21:01
Thank you David & Roman. So you means updateQueue will not guaranteed available while TimerTrigger function tries to access even though updateQueue is just specific to single function ? — tt0206, Jan 15 '21 at 21:50
Correct. A single function in code does not a single runtime function make :). — David L, Jan 15 '21 at 21:55
@DavidL I have created POC for azure storage queue and it works really good with small count of messages. I have one question. how can I process thousands of messages because its taking 1+ min to add 1k messages to the queue and my requirement is to push 10k messages in each request. any suggestions will be really appreciated.. — tt0206, Jan 17 '21 at 03:58
Your underlying storage and network will impact latency. But if you need massive throughput, you should either change your message payload to include multiple items (since it just looks like guids) or you’d need something with significantly greater throughout like an azure service bus. Regardless, you need to carefully balance message payload size with message count. — David L, Jan 17 '21 at 05:20
If the comments answer OP's question, do you(@DavidL) mind post an answer in below area ? — Hury Shen, Jan 18 '21 at 06:24

score 2 · Accepted Answer · answered Jan 19 '21 at 18:48

While Azure Functions may typically share a single backplane, it is not guaranteed. Resources can be spun down or up at any time and new copies of functions may not have access to the original state. As a result, if you use static fields to share data across function executions, it should be able to reload the data from an external source.

That said, this is also not necessarily preferable due to how Azure Functions are designed to be used. Azure Functions enable high-throughput via dynamic scalability. As more resources are needed to process the current workload, they can be provisioned automatically to keep throughput high.

As a result, doing too much work in a single function execution can actually interfere with overall system throughput, since there is no way for the function backplane to provision additional workers to handle the load.

If you need to preserve state, use a form of permanent storage. This could take the form of a Azure Durable Function, an Azure Storage Queue, an Azure Service Bus queue, or even a database. In addition, in order to best take advantage of your function's scalability, try to reduce the workload to manageable batches that allow for large amounts of parallel processing. While you may need to frontload your work in a single operation, you want the subsequent processing to be more granular where possible.

ConcurrentQueue dequeue problem in Azure function

1 Answers1