We do something like this to track running pipelines and manage execution concurrency. I find Logic Apps and Azure Functions great tools for creating these kinds of solutions. Here is a rough outline of how we handle this:
- A set of Azure Functions (AF) that leverage the
Microsoft.Azure.Management.DataFactory SDK. The relevant code is at the bottom of this post.
- A log of pipeline executions in a SQL Server table. The table includes the PipelineId
and Status, and some other information. You would need to INSERT to this table whenever you create a pipeline. We use a separate Logic App that calls an AF to execute the pipeline using the "RunPipelineAsync" method in the code below, capture the new PipelineId (RunId), and send it to a Stored Procedure to log the PipelineId.
- A Logic App running on a recurrence trigger(every 3 minutes) that
a) calls a Stored Procedure that polls the table (#2 above) and returns all pipelines with Status = "InProgress";
b) foreach over the returned list and call an AF (#1 above) that checks the current status of the pipeline using the "GetPipelineInfoAsync" method in the code below;
and
c) calls another Stored Procedure to update the status in the table.
You could do something similar to this and use the "DurationInMS" to generate appropriate actions based on status = "InProgress" and total running time > {desired alert threshold}.
Here is the DataFactoryHelper class I use:
using Microsoft.IdentityModel.Clients.ActiveDirectory;
using Microsoft.Rest;
using Microsoft.Azure.Management.ResourceManager;
using Microsoft.Azure.Management.DataFactory;
using System.Collections.Generic;
using System.Threading.Tasks;
namespace AzureUtilities.DataFactory
{
public class DataFactoryHelper
{
private ClientCredential Credentials { get; set; }
private string KeyVaultUrl { get; set; }
private string TenantId { get; set; }
private string SubscriptionId { get; set; }
private DataFactoryManagementClient _client = null;
private DataFactoryManagementClient Client
{
get {
if (_client == null)
{
var context = new AuthenticationContext("https://login.windows.net/" + TenantId);
AuthenticationResult result = context.AcquireTokenAsync("https://management.azure.com/", Credentials).Result;
ServiceClientCredentials cred = new TokenCredentials(result.AccessToken);
_client = new DataFactoryManagementClient(cred) { SubscriptionId = SubscriptionId };
}
return _client;
}
}
public DataFactoryHelper(string servicePrincipalId, string servicePrincipalKey, string tenantId, string subscriptionId)
{
Credentials = new ClientCredential(servicePrincipalId, servicePrincipalKey);
TenantId = tenantId;
SubscriptionId = subscriptionId;
}
public async Task<string> RunPipelineAsync(string resourceGroupName,
string dataFactoryName,
string pipelineName,
Dictionary<string, object> parameters = null,
Dictionary<string, List<string>> customHeaders = null)
{
var runResponse = await Client.Pipelines.CreateRunWithHttpMessagesAsync(resourceGroupName, dataFactoryName, pipelineName, parameters: parameters , customHeaders: customHeaders);
return runResponse.Body.RunId;
}
public async Task<object> GetPipelineInfoAsync(string resourceGroup, string dataFactory, string runId)
{
var info = await Client.PipelineRuns.GetAsync(resourceGroup, dataFactory, runId);
return new
{
RunId = info.RunId,
PipelineName = info.PipelineName,
InvokedBy = info.InvokedBy.Name,
LastUpdated = info.LastUpdated,
RunStart = info.RunStart,
RunEnd = info.RunEnd,
DurationInMs = info.DurationInMs,
Status = info.Status,
Message = info.Message
};
}
}
}
