I am new to Azure Pipeline, and trying to create a job to create Azure ML resources.
Because the CI pipeline runs in a self-hosted agent, and can't run the pipeline in a container, I have to call docker run
command in AzureCLI@2 task. something like:
pool:
name: 'self-hosted-agent'
stages:
- stage: 'CI'
jobs:
- job: create_ml_resources
steps:
- task: AzureCLI@2
inputs:
azureSubscription: '$(WORKSPACE_SVC_CONNECTION)'
scriptType: 'bash'
scriptLocation: inlineScript
workingDirectory: $(Build.SourcesDirectory)
inlineScript: |
set -e # fail on error
docker run -e SUBSCRIPTION_ID --rm -t -v "$PWD":/usr/src -w /usr/src $(DOCKER_IMAGE) python -m python.script.to.build_resources
and build_resources.py script is like:
from azure.identity import DefaultAzureCredential
from azure.ai.ml import MLClient
ml_client = MLClient(
DefaultAzureCredential(), subscription_id, resource_group, workspace
)
**Problem: ** The problem is that I need to authenticate to ARM from within the container, so DefaultAzureCredential() can use the credential. But I dont know how to pass the credential from the pipeline job environment to the container!
Can someone shed light on how this credential can be passed to the container run?
**What I have tried so far: **
On my localhost I can fix this by running the container with
-v ~/.azure:/root/azure
, so credential, acquired by "az login", will available inside the container. But in Azure pipeline there isn't such AccessToken in~/.azure
!I also try exporting
System.AccessTone
as a environment variable, something like:
docker run -e SUBSCRIPTION_ID -e SYSTEM_ACCESSTOKEN=$(System.AccessToken) --rm -t -v "$PWD":/usr/src -w /usr/src $(DOCKER_IMAGE) bash -c 'az account list'
But no luck!