0

I am creating c# program and want to execute it from custom activity azure data factory. However, I am not getting the steps that I should follow. I have followed a Microsoft site for the same, but the steps are not clear. So please help.

AndrewGB
  • 16,126
  • 5
  • 18
  • 49
Dpp
  • 1
  • 2
  • need something step by step code by which i can configure and connect custom activity from c# program – Dpp Jul 06 '21 at 06:52
  • This answer discusses using Azure Batch (custom activity) in ADF: https://stackoverflow.com/questions/59938374/converting-xml-files-to-json-or-csv/59938808#59938808 – Joel Cochran Jul 06 '21 at 16:53
  • In this Question it is mentioned "I wrote a .NET console app and deployed it to Azure Batch" this is the part where i am stuck. how to deploy our console app into azure batch and run that app from curstom activity pipeline. Any help will be much appreciated. – Dpp Jul 07 '21 at 05:14
  • I'm just trying to be helpful, but the question does NOT say that. It says you "are creating a c# program and want to execute it from custom activity". It does not say it is a console app, nor does it say it was "deployed" to Azure Batch. I've added an answer below that will hopefully get you started. – Joel Cochran Jul 07 '21 at 13:17

1 Answers1

0

The deployment happens at runtime. Basically, Data Factory passes the executable to the Batch service. If you haven't already done so, create an Azure Batch Linked Service to your Batch Account and reference it in the Custom Activity's "Azure Batch" tab.

You will need to load the executable package to a folder in Azure Blob Storage. Make sure to include the EXE and any dependent DLLs. In the "Settings" tab, do the following:

  1. Reference the Blob Storage Linked Service
  2. Reference the folder path that holds the executable(s).
  3. Specify the command to execute (which should be the ConsoleAppName.exe).

Here is a screen shot of the Settings:

enter image description here

If you need to pass parameters from ADF to Batch, they are called "Extended properties", and are handled differently in your Console app than typical parameters. More information can be found at this answer.

Joel Cochran
  • 7,139
  • 2
  • 30
  • 43
  • I tried all the steps and finally getting this error " Hit unexpected exception, please retry. If the problem persists, please contact Azure Support". Pipeline Run ID - c60c9834-d70a-43e7-b7c2-7927beed95bc Integration runtime - DefaultIntegrationRuntime (East US) – Dpp Jul 08 '21 at 06:46
  • Batch runs can be difficult to diagnose. This error could mean there is a problem with Data Factory, the Batch Account, or the code itself. In the Batch Account, under the Pool, you should be able to find the failing Task. At the bottom of the Task panel should be links to stdout.txt and stderr.txt. They may have more information you can pursue. – Joel Cochran Jul 08 '21 at 12:52
  • There is also a desktop app "Azure Batch Explorer" that you may find useful: https://azure.github.io/BatchExplorer/ – Joel Cochran Jul 08 '21 at 12:52