Azure Functions EventGrid trigger

Question

I have implemented an EventGrid Trigger to respond to Blob Storage Events the logic of which is simplified below :

public static async void Run(
    JObject eventGridEvent,
    TraceWriter log,
    ExecutionContext context)
{
    string eventContent = ParseEvent(eventGridEvent);

    HttpClient client = GetProxyClient();
    HttpResponseMessage response = await client.GetAsync("blabla/" + eventContent);
    string responseContent = await response.Content.ReadAsStringAsync();
    log.Info("Here is the response :" + responseContent);
}

The external API does not take long to respond (1 second or less) and my configuration for the host is set to default (so an unbounded number of concurrent calls is allowed).

I am getting a lot of duplicated events in the logs when adding multiple blobs (starting at just 2 blobs) at the same time (a script is quickly uploading the blobs one by one with no wait time in between).

I feel that this might be due to the fact that I never acknowledge receiving the events and I don't know if I am supposed to do this in my code or whether the EventGrid Trigger does that automatically.

Is the logic for acknowledging the processing of an event supposed to be implemented within an EventGrid Trigger (Http 200 response) or is this handled automatically?

If not should I still be getting duplicated events? Typically, when uploading a single blob I receive the event for it 3-4 times.

The reason I ask this question is that when using a Http Trigger and returning a 400 response I also get duplicated events which makes sense since I am not acknowledging having correctly processed the event. However, when I return a 200 response then I do not receive duplicated events.

Thanks

I too see the same issue with Azure Functions event grid trigger. A single blob insertion triggers an Azure Function 3-4 times. Do we have any update on this? — No8, Mar 21 '19 at 11:34
I'm seeing the exact same behaviour. How do we solve this for longer running Functions? — ericOnline, Nov 03 '20 at 19:23

score 5 · Answer 1 · answered May 16 '18 at 11:55

5

You don't need to do anything special to indicate success to Event Grid. If your function execution succeeds (does not throw exception), the trigger will respond with success status code automatically.

answered May 16 '18 at 11:55

Mikhail Shilkov

34,128
3
68
107

Just to be clear on this, if my Function takes 3 minutes to execute, but the EventGrid retry policy is set to 30secs, EventGrid will continue to retry until the Function completes, correct? – ericOnline Nov 03 '20 at 19:22
Is there a way to ack the EventGrid message earlier in the code to prevent this retry behaviour? – ericOnline Nov 03 '20 at 19:22
This is correct. I don't think you can ack earlier, so you should keep your functions short (e.g., offload heavy work to a queue-based function). – Mikhail Shilkov Nov 03 '20 at 19:47
Hm. Is there a way to adjust the EventGrid retry time-to-wait? – ericOnline Nov 03 '20 at 20:52
Strangely enough, this behavior does not occur at all in my DEV Function. Only PROD. Both are moving large files/running same durations. – ericOnline Nov 03 '20 at 20:54
[I'm seeing inconsistent EventGrid retries for Functions that run in ~1 - 4.5secs](https://stackoverflow.com/questions/64669555/eventgrid-triggered-python-azure-function-keeps-triggering-after-successfully-r/64675376#64675376). Is there some kind of response latency between Azure Function and EventGrid? – ericOnline Nov 04 '20 at 17:55

score 1 · Answer 2 · answered May 26 '21 at 19:10

You may try using an EventGrid Advanced Filter of data.api String ends with FlushWithClose. The reason my Azure Function was executing multiple times upon blob upload was because an EventGrid message was created for every AppendFile action being performed for the blob upload.

I found out (by trial and error) that Azure Data Factory uses a series of API calls to write a single blob to Blob Storage.

Ends up looking something like this:

CreateFilePath
LeaseFile
AppendFile
AppendFile
AppendFile (each append puts a chunk of the blob until the blob is complete)
FlushFile (this is the actual indication that the file has finished; hence the Advanced Filter shown above)
LeaseFile

Here is a sample query to view this upload flow yourself:

Note: You'll need the Uri of a sample file uploaded to the blob container

//==================================================//
// Author: Eric
// Created: 2021-05-26 0900 
// Query: ADF-to-Blob Storage reference flow
// Purpose: 
// To provide a reference flow of ADF-to-Blob Storage
// file uploads
//==================================================//
// Assign variables
//==================================================//
let varStart = ago(10d);
let varEnd = now();
let varStorageAccount = '<storageaccountname>';
let varStatus = 'Success';
let varSampleUri = 'https://<storageaccountname>.dfs.core.windows.net/<containername>/<parentfolder1>%2F<parentfolder2>%2F<samplefilename.extension>'
//==================================================//
// Filter table
//==================================================//
StorageBlobLogs
| where TimeGenerated between (varStart .. varEnd)
  and AccountName == varStorageAccount
  and StatusText == varStatus
  and split(Uri, '?')[0] == varSampleUri
//==================================================//
// Group and parse results
//==================================================//
| summarize 
  count() by OperationName,
  CorrelationId,
  TimeGenerated,
  UserAgent = tostring(split(UserAgentHeader, ' ')[0]),
  RequesterAppId,
  AccountName, 
  ContainerName = tostring(split(tostring(parse_url(url_decode(Uri))['Path']), '/')[1]),
  FileName = tostring(split(tostring(parse_url(url_decode(Uri))['Path']), '/')[-1]),
  ChunkSize = format_bytes(RequestBodySize, 2, 'MB'),
  StatusCode,
  StatusText
| order by TimeGenerated asc

Its interesting to upload samples from different sources (Azure Data Factory, Azure Storage Explorer, Python/C# SDK, Azure Portal, etc.) and see the different API methods they use. In fact, you'll likely need to do this to get your logging and alerting dialed in.

Its too bad the methods aren't standardized across tools as this particular issue is a great pain to discover on your own!

Again, EventGrid Advanced Filters are your friend in this case.

Azure Functions EventGrid trigger

2 Answers2