Azure ML Studio Pipeline is run under Service Principle

Question

I have a draft pipeline created inside the azure machine learning service workspace (Designer Mode). I try to run pipeline from python using Python Azure ML SDK. It starts but quickly fails on the second step.

Trace from step:

Traceback (most recent call last):
  File "invoker.py", line 81, in <module>
    execute(args)
  File "invoker.py", line 71, in execute
    ret = run(generate_run_command(args))
  File "invoker.py", line 52, in run
    return subprocess.Popen(command, stdout=sys.stdout, stderr=sys.stderr).wait(timeout=timeout)
  File "/azureml-envs/azureml_b05af1507517824d92fd90bb8ce7897a/lib/python3.8/subprocess.py", line 858, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/azureml-envs/azureml_b05af1507517824d92fd90bb8ce7897a/lib/python3.8/subprocess.py", line 1704, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
PermissionError: [Errno 13] Permission denied: ''

When I submit a job for the drafted pipeline in UI there is no problem. When I submit a job for the same draft pipeline from Python SDK, it fails with "Permission denied" on the second step, which actually "Apply SQL Transformation", the first step is Import Dataset. When I resubmit the failed job from UI there is also no problem. It is clear that the problem is in Service Principle. I granted all possible permissions to SP for the workspace. It didn't help. Does anybody have luck running Azure ML Drafted Pipeline from Python?

score 1 · Answer 1 · answered Dec 01 '22 at 13:58

Service principal method works for me when using Azure ML Drafted Pipeline from Python. I am Using-python 3.7 and azureml-sdk 1.47.0

My code

from azureml.core import Workspace
from azureml.core.authentication import ServicePrincipalAuthentication
from dotenv import load_dotenv
load_dotenv()

def getMLWorkspace():

    # Connect to the Azure ML Service Workspace using a service principal
    svcpr = ServicePrincipalAuthentication(
    tenant_id = os.environ['TENANT_ID'],
    service_principal_id = os.environ['SERVICE_PRINCIPAL_ID'],
    service_principal_password = os.environ['SERVICE_PRINCIPAL_PWD'])

    subscription_id = os.environ['SUBSCRIPTION_ID']
    resource_group = os.environ['RESOURCE_GROUP']
    workspace_name = os.environ['WORKSPACE_NAME']
    
    ws = Workspace(
    subscription_id = subscription_id,
    resource_group = resource_group,
    workspace_name = workspace_name,
    auth = svcpr)
    
    print('Workspace configuration succeeded')
    return ws

Azure ML Studio Pipeline is run under Service Principle

1 Answers1