1

I am configuring an Environment in azureml based on a conda enviroment file. The azureml environment seems to be ignoring the enviromnet variables however.

from azureml.core import Environment
from azureml.core.conda_dependencies import CondaDependencies

CondaDependencies._VALID_YML_KEYS.append("variables")
pipeline_env = Environment.from_conda_specification("pipeline_env", "env.yml")
print(pipeline_env.environment_variables)

This results in the following being printed.

{'EXAMPLE_ENV_VAR': 'EXAMPLE_VALUE'}

My env.yml contain the follow section at the bottom

variables:
- KEY_ONE: 1.1.0.1
- KEY_TWO: 1.1.0.1

And if i save my environment to directory like this

pipeline_env.save_to_directory("env")

it produces a folder named "env" which contain two files.

  • conda_dependencies.yml
  • azureml_environment.json

In the azureml_environment i can see that my two keys do not exist. They do however exist in the conda_dependancies.yml which indicate to me that they are correctly defined in the env.yml file.

I also had to add the "varialbes" key as a valid yml key as shown, if not azureml threw an error.

I am starting to suspect that azureml does not allow this method of setting the environment variables, and that the only way to set them correctly is to use the following method:

 pipeline_env.environment_variables = {"KEY_ONE", "1.1.0.1",
                                       "KEY_TWO", "1.1.0.1")

As this does work, i would prefer to use the .yml file however. So i guess my question is: Should i be able to set environment variables using the .yml file, or is my assumption correct that i have to use the enviroment_variables function?

VegardKT
  • 1,226
  • 10
  • 21

1 Answers1

2

environment_variables in environment definition are deprecated and originally runtime variables set for the job on compute target and not baked into container. The default value example_env_var:example_value is still there for backward compatibility but will be removed eventually.

https://learn.microsoft.com/en-us/python/api/azureml-core/azureml.core.environment.environment?view=azure-ml-py#azureml-core-environment-environment-environment-variables

variables set in the yml file should be passed to the conda create during the image materialization

vizhur
  • 467
  • 3
  • 7
  • "environment_variables in environment definition are deprecated and originally runtime variables set for the job on compute target and not baked into container." and "variables set in the yml file should be passed to the conda create during the image materialization" are these two not contradicting? – VegardKT Feb 03 '22 at 16:26
  • 1
    not at all. variables that mistakenly (AzureML takes responsibility for that) expected in the container from environment_variables property correspond to `ENV MYVAR` docker command, anything specified in the conda spec is passed to `conda create` as is and that is between user and conda logic – vizhur Feb 04 '22 at 19:16