1

I am new in Azure and I have a problem with Function App. I hope you can help me.
I have an Azure function with a blob trigger that runs for every file that is placed in Azure Datalake Gen2 storage. If there is a failure, I don't want retries.
I use the function to process 400 files that I put manually in Datalake using Microsoft Azure Storage. Then, my function should run 400 times to process all documents. But it never process all documents. Sometimes process 95, others 295.
I have search for information of Azure App Funtions limitation in number of times they will run but I found no information.
I paste json files below
function.json

{
  "scriptFile": "__init__.py",
  "bindings": [
    {
      "name": "inStr",
      "type": "blobTrigger",
      "direction": "in",
      "path": "container/input/{name}",
      "connection": "AzureWebJobsStorage"
    }
  ]
}

I have more function App, with one function, and I use same connection "AzureWebJobsStorage". May the problem be here?
The connection string "AzureWebJobsStorage" format is

"AzureWebJobsStorage": "DefaultEndpointsProtocol=https;AccountName=mydlaccount;AccountKey=key;EndpointSuffix=core.windows.net",

host.json

{
  "version": "2.0",
  "logging": {
    "applicationInsights": {
      "samplingSettings": {
        "isEnabled": true
      }
    }
  },
  "extensions": {
    "blobs": {
        "maxDegreeOfParallelism": "4"
    },
    "queues": {
      "maxDequeueCount": 1
    }
  },
  "extensionBundle": {
    "id": "Microsoft.Azure.Functions.ExtensionBundle",
    "version": "[2.*, 3.0.0)"
  },
  "retry": {
    "strategy": "fixedDelay",
    "maxRetryCount": 0,
    "delayInterval": "00:00:05"
  }
    
}
vll1990
  • 311
  • 3
  • 17
  • Are you running in Consumption tier or any other? How long your file processing logic would take to complete for a file - give average ? Any function should finish within certain time, for e.g max timeout is 5 mins for consumption tier. – Anand Sowmithiran Dec 14 '21 at 14:41
  • Hi @AnandSowmithiran. Thanks for your answer. Yes, the function is running in Consumption tier and the time to process each document is less than 1 minute. The problem is that function is not triggered by all input files and I don't see any information of why – vll1990 Dec 15 '21 at 08:22
  • Have you tried increasing the `maxDegreeOfParallelism` host.json parameter? How are the files copied to your blob storage - via script or external programs or Azure Data factory, etc? – Anand Sowmithiran Dec 15 '21 at 09:06
  • https://i.imgur.com/A1g1VCf.png - I too have 400 text files named sequentially and when I tried to upload them all at once manually to ADLS Gen 2 Storage Account, they're uploaded successfully but processing is happening randomly like this https://i.imgur.com/xQRQ1Ft.png - @vll1990 Is the processing of files happening randomly or sequentially for you? –  Dec 19 '21 at 04:45
  • Hi Anand Sowmithiran, thanks for your answer. I tried to increase maxDegreeOfParallelism but the behavior is the same. Files are copied by a middleware, an external program. I have other process that copies them with Azure Data Factory and in this case the trigger doesn't work. – vll1990 Dec 27 '21 at 17:04
  • Hi HariKrishnaRajoli-MT. I think they are processed randomly, but in my case this is not a problem. As I bulk copied them, I don't know the order they arrive to the datalake and I suppose they are processed in this arrival order – vll1990 Dec 27 '21 at 17:06
  • I am facing same issues, even i am uploading files manually around 50, it just process 28 files only. I have checked the Log Stream, its just read the file but not process them like other files. Could you please help me if you have resolved this issue. – Shubham Tomar Oct 14 '22 at 10:24

0 Answers0