2

My azure function is calculating results of certain request jobs (cca. 5s-5min) where each job has unique jobId based on the hash of the request message. Execution leads to deterministic results. So it is functionally "pure function". Therefore we are caching results of already evaluated jobs in a blob storage based on the jobId. All great so far.

Now if a request for jobId comes three scenarios are possible.

  1. Result is in the cache already => then it is served from the cache.
  2. Result is not in the cache and no function is running the evaluation => new invocation
  3. Result is not in the cache, but some function is already working on it => wait for result

We do some custom table storage based progress tracking magic to tell if function is working on given jobId or not yet.

It works somehow, up to the point of 5 x restart -> poison queue scenarios. There we are quite hopeless.

I feel like we are hacking around some of already reliably implemented feature of Azure Functions internals, because exactly the same info can be seen in the monitor page in azure portal or used to be visible in kudu webjobs monitor page.

How to reliably find out in c# if a given message (jobId) is currently being processed by some function and when it is not?

Janusz Nowak
  • 2,595
  • 1
  • 17
  • 36
Richard Vondracek
  • 97
  • 6

2 Answers2

2

Azure Durable Functions provide a mechanism how to track progress of execution of smaller tasks.

https://learn.microsoft.com/en-us/azure/azure-functions/durable-functions-overview

Accroding to the "Pattern #3: Async HTTP APIs" the orchestrator can provide information about the function status in form like this:

  {"runtimeStatus":"Running","lastUpdatedTime":"2017-03-16T21:20:47Z", ...}

This solves my problem about finding if given message is being processed.

Richard Vondracek
  • 97
  • 6
  • Thanks Richard for posting your solution. Would you mind marking it as the accepted answer? That will make it easier for other users to find it. – Chris Gillum Jan 04 '19 at 04:37
0

How to reliably find out in c# if a given message (jobId) is currently being processed by some function and when it is not?

If you’d like to detect which message is being processed and get the message ID in queue triggered Azure function, you can try the following code:

#r "Microsoft.WindowsAzure.Storage"
using System;
using Microsoft.WindowsAzure.Storage.Queue; 

public static void Run(CloudQueueMessage myQueueItem, TraceWriter log)
{
    log.Info($"messageid: {myQueueItem.Id}, messagebody: {myQueueItem.AsString}");
}
Fei Han
  • 26,415
  • 1
  • 30
  • 41
  • Thanks for your effort, but It seems like you are suggesting a function that processes the message from queue. I already have a function that processes a message "FunctionA" and I want other function "FunctionB" to find out if some "FunctionA" is already working on the message or not. It makes sense for "FunctionB" to wait for the result if the process started rather than queueing a new message. – Richard Vondracek Oct 18 '17 at 08:45