If we have PyPi Packages added as Artifacts to an Azure DevOps Project Feed, how can we use these packages as a source for installing packages in DatabricksStep
of Azure Machine Learning Service?
While using pip
in any environment, we use our Azure DevOps Project Artifacts feed in the following way:
pip install example-package --index-url=https://<Personal-Access-Token>@pkgs.dev.azure.com/<Organization-Name>/_packaging/<Artifacts-Feed-Name>/pypi/simple/
The DatabricksStep class of the Azure Machine Learning Service accepts the following parameters:
python_script_name = "<Some-Script>.py"
source_directory = "<Path-To-Script>"
<Some-Placeholder-Name-for-the-step> = DatabricksStep(
name=<Some-Placeholder-Name-for-the-step>,
num_workers=1,
python_script_name=python_script_name,
source_directory=source_directory,
run_name= <Name-of-the-run>,
compute_target=databricks_compute,
pypi_libraries = [
PyPiLibrary(package = 'scikit-learn'),
PyPiLibrary(package = 'scipy'),
PyPiLibrary(package = 'azureml-sdk'),
PyPiLibrary(package = 'joblib'),
PyPiLibrary(package = 'azureml-dataprep[pandas]'),
PyPiLibrary(package = 'example-package', repo='https://<Personal-Access-Token>@pkgs.dev.azure.com/<Organization-Name>/_packaging/<Artifacts-Feed-Name>/pypi/simple/')
],
allow_reuse=True
)
However, PyPiLibrary(package = 'example-package', repo='https://<Personal-Access-Token>@pkgs.dev.azure.com/<Organization-Name>/_packaging/<Artifacts-Feed-Name>/pypi/simple/')
will give an error. How exactly should we consume the Artifacts Feed as an input to the PyPiLibrary
property of the DatabricksStep
Class in Azure Machine Learning Service?