1

Using Azure Data Factory V2 I have 2 piplelines. One that reads CSV data files uploaded to a blob storage container & processes the data. That pipeline is triggered by "Blob Created". I have set up "Warning" Level Logging for pipeline 1 to catch data errors, missing column headers, extra (un-mapped) columns, and I created the destination folder to store those log files called 'file-data-errors'. That all works. The 'file-data-errors' container is being populated with *.txt files as expected.

I want to populate an Azure SQL Table with the contents of the CSV files. To do that I set up another pipeline with a trigger that fires on "Blob Created" when a file is added to the 'file-data-errors' container.

To test the 2nd pipleline, I upload a file (one earlier created by pipeline 1) to the 'file-data-errors' container, the process succeeds.

But when I upload a file to the blob storage of the first pipeline, the second pipeline does not get triggered as expected. That is, a file is added to the 'file-data-errors' container as normal, but the "Blob Created" trigger doesn't fire.

I have found a very similar question on stackoverflow, Azure Storage V2 blob event - Not triggering ADF but the suggested solution does not work for me

What am I missing?

RW-Fairport
  • 61
  • 6

1 Answers1

1

Just post an answer to end this question.

Solution:

By changing the 2nd trigger to a schedule instead of the blob created event, I got it working. The 2nd pipleline only needs to run daily anyway, so this is working OK now

Steve Johnson
  • 8,057
  • 1
  • 6
  • 17