-1

I am using a Python code that creates the batch pool using the Azure SDK for Python. I have deployed this code in function apps and I am using it in data factory to create the batch pool. I want to check if the nodes have started or if they are in idle state before I run the rest of components in the pipeline. Is there any way I can check this in data factory?

Sindu_
  • 1,347
  • 8
  • 27
  • 67

2 Answers2

0

No, there isn't a way can check it in Data Factory. Data Factory will call and run the Azure function directly.

Leon Yue
  • 15,693
  • 1
  • 11
  • 23
  • Is there any way to check using the Python script after creating the resources? – Sindu_ Jun 22 '21 at 10:11
  • 1
    @Sindu_ It sounds achievable. Data Factory pipeline may like this: Azure function-->python. Function active to create the batch pool and python active to check if the nodes have started. – Leon Yue Jun 23 '21 at 00:52
-1

Yes, there is no direct way to check the node status via ADF. But you can consider below workaround to achieve the same via ADF using Web Activity.

  • Step 1: Make an API to call Azure Batch services to get to your node status details. We can leverage web activity in Azure data factory to do the same.

  • Step 2: Based on response from Step 1, Invoke execution of your pipeline further activities.

Check below links to know more details about Batch Service API and Web activity

IpsitaDash-MT
  • 1,326
  • 1
  • 3
  • 7