Hope you are doing well.
I am currently facing an issue with the execution of my training model pipelines(pipeline.yml for Github and "deploy-model-training-pipeline-classical.yml" for Azure-DevOps) from my ML project set with Mlops-v2 accelerator
From Github: This pipeline is supposed to execute a number of jobs.As we can in the picture below most of the jobs have been correctly executed except the
"register-dataset".
The error log specifies /home/runner/xxxxxx/xxxx/mlops/azureml/train/data/data.yml this path cannot be found.
I have tried different way to fix that i.e moving the data.yml into root folder , modifying the path of the data.yml in the "deploy-model-training-pipeline-classical.yml" file but I am still get the same error mentioned the same error /home/runner/xxxxxx/xxxx/mlops/azureml/train/data/data.yml alghouth I changed the path in the pipeline file and the moved the data.yml in the root.log
Is there anything I need to the update to make it work?
Below you will see the content of the register YAML file for Github.
name: deploy-model-training-pipeline
on:
workflow_dispatch:
jobs:
set-env-branch:
runs-on: ubuntu-latest
outputs:
config-file: ${{ steps.set-output-defaults.outputs.config-file }}
steps:
- id: set-prod-branch
name: set-prod-branch
if: ${{ github.ref == 'refs/heads/main'}}
run: echo "config_env=config-infra-prod.yml" >> $GITHUB_ENV;
- id: set-dev-branch
name: setdevbranch
if: ${{ github.ref != 'refs/heads/main'}}
run: echo "config_env=config-infra-dev.yml" >> $GITHUB_ENV;
- id: set-output-defaults
name: set-output-defaults
run: |
echo "config-file=$config_env" >> $GITHUB_OUTPUT;
get-config:
needs: set-env-branch
uses: Azure/mlops-templates/.github/workflows/read-yaml.yml@main
with:
file_name: ${{ needs.set-env-branch.outputs.config-file}}
register-environment:
needs: get-config
uses: Azure/mlops-templates/.github/workflows/register-environment.yml@main
with:
resource_group: ${{ needs.get-config.outputs.resource_group }}
workspace_name: ${{ needs.get-config.outputs.aml_workspace }}
environment_file: mlops/azureml/train/train-env.yml
conda_file: data-science/environment/train-conda.yml
secrets:
creds: ${{secrets.AZURE_CREDENTIALS}}
register-dataset:
needs: get-config
uses: Azure/mlops-templates/.github/workflows/register-dataset.yml@main
with:
resource_group: ${{ needs.get-config.outputs.resource_group }}
workspace_name: ${{ needs.get-config.outputs.aml_workspace }}
name: UpdateRocheDB
data_file: mlops/azureml/train/data.yml
secrets:
creds: ${{secrets.AZURE_CREDENTIALS}}
create-compute:
needs: [get-config]
uses: Azure/mlops-templates/.github/workflows/create-compute.yml@main
with:
cluster_name: cpu-cluster
size: Standard_DS3_v2
min_instances: 0
max_instances: 4
cluster_tier: low_priority
resource_group: ${{ needs.get-config.outputs.resource_group }}
workspace_name: ${{ needs.get-config.outputs.aml_workspace }}
secrets:
creds: ${{secrets.AZURE_CREDENTIALS}}
run-model-training-pipeline:
needs: [get-config, register-environment, register-dataset, create-compute]
uses: Azure/mlops-templates/.github/workflows/run-pipeline.yml@main
with:
resource_group: ${{ needs.get-config.outputs.resource_group }}
workspace_name: ${{ needs.get-config.outputs.aml_workspace }}
parameters-file: mlops/azureml/train/pipeline.yml
job-name: test
secrets:
creds: ${{secrets.AZURE_CREDENTIALS}}
From Azure:
the pipeline yaml syntax seems not to be recognized by Azure-devops(see the imgae enter image description here.
$schema: https://azuremlschemas.azureedge.net/latest/pipelineJob.schema.json
type: pipeline
experiment_name: fish-health-diagnostics-training
experiment_name: fish-health-diagnostics-training
description: Training Pipeline to train model 0 used to diagnostic the fish health
# <inputs_and_outputs>
inputs:
input:
type: uri_file
path: azureml:fish-health-train-env@latest
enable_monitoring: 'false'
table_name: 'fishhealthmonitoring'
outputs:
train_data:
val_data:
test_data:
trained_model:
evaluation_output:
model_info_output_path:
# </inputs_and_outputs>
# <jobs>
settings:
default_datastore: azureml:workspaceblobstore
default_compute: azureml:cpu-cluster
continue_on_step_failure: false
jobs:
prep_data:
name: prep_data
display_name: prep-data
code: ../../../data-science/src
command: >-
python prep.py
--raw_data ${{inputs.raw_data}}
--train_data ${{outputs.train_data}}
--val_data ${{outputs.val_data}}
--test_data ${{outputs.test_data}}
--enable_monitoring ${{inputs.enable_monitoring}}
--table_name ${{inputs.table_name}}
environment: azureml:fish-health-train-env@latest
inputs:
raw_data: ${{parent.inputs.input}}
enable_monitoring: ${{parent.inputs.enable_monitoring}}
table_name: ${{parent.inputs.table_name}}
outputs:
train_data: ${{parent.outputs.train_data}}
val_data: ${{parent.outputs.val_data}}
test_data: ${{parent.outputs.test_data}}
train_model:
name: train_model
display_name: train-model
code: ../../../data-science/src
command: >-
python train.py
--train_data ${{inputs.train_data}}
--model_output ${{outputs.model_output}}
environment: azureml:fish-health-train-env@latest
inputs:
train_data: ${{parent.jobs.prep_data.outputs.train_data}}
outputs:
model_output: ${{parent.outputs.trained_model}}
evaluate_model:
name: evaluate_model
display_name: evaluate-model
code: ../../../data-science/src
command: >-
python evaluate.py
--model_name ${{inputs.model_name}}
--model_input ${{inputs.model_input}}
--test_data ${{inputs.test_data}}
--evaluation_output ${{outputs.evaluation_output}}
environment: azureml:fish-health-train-env@latest
inputs:
model_name: "wellfish-model0"
model_input: ${{parent.jobs.train_model.outputs.model_output}}
test_data: ${{parent.jobs.prep_data.outputs.test_data}}
outputs:
evaluation_output: ${{parent.outputs.evaluation_output}}
register_model:
name: register_model
display_name: register-model
code: ../../../data-science/src
command: >-
python register.py
--model_name ${{inputs.model_name}}
--model_path ${{inputs.model_path}}
--evaluation_output ${{inputs.evaluation_output}}
--model_info_output_path ${{outputs.model_info_output_path}}
environment: azureml:fish-health-train-env@latest
inputs:
model_name: "wellfish-model0"
model_path: ${{parent.jobs.train_model.outputs.model_output}}
evaluation_output: ${{parent.jobs.evaluate_model.outputs.evaluation_output}}
outputs:
model_info_output_path: ${{parent.outputs.model_info_output_path}}
# </jobs>
