1

I'm trying to start a training job on Azure ML from an Azure function (blob trigger) using Python. Here're the two ways I've seen it done:

  1. Using the command function from azure.ai.ml
    inputs=dict(
        data_folder=Input(type="uri_folder", path=web_path)
    ),
    compute=gpu_compute_target,
    environment=curated_env_name,
    code="./src/",
    command="python tf_mnist.py --data-folder ${{inputs.data_folder}}",
    experiment_name="tf-dnn-image-classify",
    display_name="tensorflow-classify-mnist-digit-images-with-dnn",
)
ml_client.jobs.create_or_update(job)
  1. Using ScriptRunConfig class from azureml.core
                      script='train.py', 
                      arguments=['--input-data-dir', dataset.as_named_input('input').as_mount(),
                                 '--reg', '0.99'],
                      run_config=run_config) 
run = experiment.submit(config=src)

What is the difference between these? My source code is in a remote repo and this bit of code would be running in an Azure function. Which of these methods is suitable for my scenario?

Judy T Raj
  • 1,755
  • 3
  • 27
  • 41

1 Answers1

1

They key difference between those methods is just the version of the SDK/API that they are using.

  • Code from azure.ai.ml is coming from the Python SDK V2, contained in the Python package azure-ai-ml. More details about, including how to install it, can be found here
  • The other code snippet you posted uses the Python SDK V1, contained in the package azureml-core. Instructions here.

I would suggest using the V2 SDK, it is clearly more future-proof (though of course you may hit some snags for now because the V2 SDK is still in preview mode). Our team are also now switching from V1 to V2.

Anton Schwaighofer
  • 3,119
  • 11
  • 24
  • Thank you, that works. Do you know if the source code has to be present locally or if I can pass a remote path to the `code` argument? The documentation seems to say local path but I'm running this on an azure function. – Judy T Raj Oct 10 '22 at 09:49
  • 1
    To the best of my knowledge, the code has to be present locally. What the SDK does under the hood is copy everything to blob storage, where it will be later read from the AzureML job. The folder to copy is given in the `code` argument in your V2 example. And note that the command to run must work relative to that code folder. – Anton Schwaighofer Oct 10 '22 at 11:47