Where do you do CallActivityAsync in orchestration method

Question

I have just started using durable functions and needs some advise for how to do fan out pattern correctly. I have a FTP server where from I read all the files. I want to start an Activity function for each file. As I understand it the orchestrator function will be called everytime an Activity function is being executed. I just want to read the files once. To avoid calling the code that read the files and starts the activity functions multiple times, what is the recommended approach? Is it having an activity function that that add's all the activity functions or is it using the IsReplaying property, or something different?

[FunctionName("OrchestrationMoveFilesToBlob")]
public static async Task<List<string>> RunOrchestrator(
    [OrchestrationTrigger] DurableOrchestrationContext context)
{
    var outputs = new List<string>();

    if (!context.IsReplaying)
    {
        // Do you call your database here and make a call to CallActivityAsync for each row?
    }

    // doing it here is properly very wrong as it will be called multiple times
    var tasks = new Task<string>[7];
    for (int i = 0; i < 7; i++)
    {
        tasks[i] = context.CallActivityAsync<string>("E2_CopyFileToBlob","");            }

    await Task.WhenAll(tasks);

    return outputs;
}

When looking into the sample in the link below this actually calls it directly in the orchestrator function? Is this not really bad? It continue adding same activities again and again .... ?

https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-cloud-backup

Answer 1

Not sure I understand what you try to achieve but your code looks not bad so far. An orchestration is just called once (and maybe some times more for replay but this is not your problem here). From your orchestration you can call in a fan out all your activity functions (gathering a file from an ftp) each activity function one file. await Task.WhenAll(tasks) is your fan in. (you can use a List<Task> instead of the array and call .Add(task) on it if you want. In order to not edit your code I copied it here and added some comments and questions (feel free to edit here):

[FunctionName("OrchestrationMoveFilesToBlob")]
public static async Task<List<string>> RunOrchestrator(
    [OrchestrationTrigger] DurableOrchestrationContext context)
{
    var outputs = new List<string>();

    if (!context.IsReplaying)
    {
        // just needed for things that should not happen twice like logging....
    } 

    // if your work isn't a fixed list just call an activity 
    // which replies with the list of work here (e.g. list of filenames)

    var tasks = new Task<string>[7]; // can be a List<Task> too
    for (int i = 0; i < 7; i++)
    {
        tasks[i] = context.CallActivityAsync<string>("E2_CopyFileToBlob","");
    }

    await Task.WhenAll(tasks);

    return outputs; // currently an empty list. What do you want to give back?
}