0

We have lots of files that get created in blob storage and consequently we are passing these files to the azure functions. However with Azure function and event grid trigger it was timed out after certain no. of files were processed. Hence now I am trying with durable functions.

Below is the code I tried.

ProcessJob_EventGrid gets triggered. However the ProcessJob_Orchestrator never gets triggered. Is there something that I am missing.

I am completely new to the Durable functions concept.

[FunctionName("ProcessJob_Orchestrator")]
        public async Task RunOrchestrator(
            [OrchestrationTrigger] IDurableOrchestrationContext context, ILogger log)
        {
            log.LogInformation($"************** RunOrchestrator method executing ********************");
            var data = context.GetInput<string>();
            log.LogInformation($"File Name is  {data}.");
            //// Replace "hello" with the name of your Durable Activity Function.
            //outputs.Add(await context.CallActivityAsync<string>("MCNDataTransformation_Hello", "Tokyo"));
            //outputs.Add(await context.CallActivityAsync<string>("MCNDataTransformation_Hello", "Seattle"));
            //outputs.Add(await context.CallActivityAsync<string>("MCNDataTransformation_Hello", "London"));

            //// returns ["Hello Tokyo!", "Hello Seattle!", "Hello London!"]
            //return outputs;
        }

        [FunctionName("MCNDataTransformation_Hello")]
        public string SayHello([ActivityTrigger] string name, ILogger log)
        {
            log.LogInformation($"Saying hello to {name}.");
            return $"Hello {name}!";
        }

        [FunctionName("ProcessJob_EventGrid")]
        public static async void EventGridTest([EventGridTrigger] EventGridEvent eventGridEvent, [DurableClient] IDurableOrchestrationClient starter, ILogger log)
        {
            JObject objData = JObject.Parse(eventGridEvent.Data.ToString());

            // Function input comes from the request content.
            string instanceId = await starter.StartNewAsync("ProcessJob_Orchestrator", null, JsonConvert.SerializeObject(objData));

            log.LogInformation(eventGridEvent.Data.ToString());
        }
Daniel Mann
  • 57,011
  • 13
  • 100
  • 120
amit agarwal
  • 63
  • 2
  • 17

1 Answers1

0

I'm a member of the Durable Functions developer team :) Nothing immediately stands out as being wrong. However, passing null as the 2nd parameter toStartNewAsync is a bit suspicious: unless you're trying to invoke a specific instanceId, there should be no reason for your to specify that value.

How about you try rewriting that line to be:

// Do not specify instanceId argument if not needed.
string instanceId = await starter.StartNewAsync("ProcessJob_Orchestrator", JsonConvert.SerializeObject(objData));

Perhaps that should do it. Please let me know if it does!

If that doesn't fix your problem, there are a few follow-up steps you can take.

  1. Locally check-out the precompiled samples here: https://github.com/Azure/azure-functions-durable-extension/tree/dev/samples/precompiled In there, try running a simple orchestration such as HelloSequence. If you manage to run it, try modifying it to fit your use-case.

  2. If none of the advise above works, open a ticket in https://github.com/Azure/azure-functions-durable-extension/issues and we'll take a look at this promptly!

Dharman
  • 30,962
  • 25
  • 85
  • 135
David Justo
  • 26
  • 2
  • The only reason for me to pass null is the second parameter is expecting instance id so I passed it as null but if I remove it then I get the below error ----------------------------------2020-11-26T07:27:55.995 [Error] Executed 'ProcessJob_EventGrid' (Failed, Id=6f1fefbe-c9e7-4550-8b47-7969226b39dc, Duration=18ms)instanceId (Parameter 'Orchestration instance ids must not contain /, \, #, ?, or control characters.') – amit agarwal Nov 26 '20 at 07:29
  • I tried with all the 3 below combinations. string instanceId = await starter.StartNewAsync("HelloWorld"); string instanceId = await starter.StartNewAsync("HelloWorld",JsonConvert.SerializeObject(objData)); -- Error string instanceId = await starter.StartNewAsync("HelloWorld",null , JsonConvert.SerializeObject(objData)); – amit agarwal Nov 26 '20 at 07:47
  • 1
    What version of `Microsoft.Azure.WebJobs.Extensions.DurableTask` are you working with? Also, are you able to locally run the sample I listed above? – David Justo Nov 26 '20 at 17:58
  • The version for Microsoft.Azure.WebJobs.Extensions.DurableTask is 2.1.1 . The sample you have provided is for Httptrigger and its already working for me too. However for event trigger is the issue where the orchestration do not get fired. – amit agarwal Nov 27 '20 at 07:23
  • I see, I get what you mean. Please consider opening an issue here: https://github.com/Azure/azure-functions-durable-extension/issues and I'll notify the rest of the team to try to get to the bottom of it! – David Justo Nov 30 '20 at 15:13
  • I was able to find out the issue. Basically , the messages were not logged but the same I tried it debugging running in my local with the help of ngrok to understand the root cause of it. But in my local it was working as expected and same when I had deployed the messages were still not getting logged. So I wrote few insert scripts to the db in my activity function . And records were getting created. But surprisibly the messages are still not getting logged. – amit agarwal Nov 30 '20 at 16:12
  • Just to clarify, you're saying that the orchestrator is actually executing after all, and that the problem is that your log-statements are not showing, right? – David Justo Nov 30 '20 at 17:26
  • Yes... but it does get logged with Ngrok. Hope I didn't confuse. – amit agarwal Dec 01 '20 at 06:27
  • Got it. In case the issue is still outstanding, please open up an issue here: https://github.com/Azure/azure-functions-durable-extension/issues and that will help me redirect the rest of the team to look at it and, hopefully, provide an explanation – David Justo Dec 07 '20 at 21:05
  • Yeah, I have already opened the issue here. https://github.com/Azure/azure-functions-durable-extension/issues/1587 – amit agarwal Dec 08 '20 at 08:18