I have an Azure ML workspace with a datastore (Datastore 1) setup maybe over 2 years ago. I created a new Azure Storage container and copied everything over from the azure storage container that serves as Datastore 1. Set this up as Datastore 2.
The file hierarchy inside the 2 containers is identical, because it was a recursive ADF copy.
Running the experiment against the 2nd datastore fails because for a particular mounted path os.path.exists returns False, but a subsequent call to os.makedirs conditioned on os.path.exists fails because:
FileExistsError: [Errno 17] File exists: '/mnt/batch/tasks/shared/LS_root/jobs/i...2/azureml/hd_8.._7/mounts/files/azureml_model/csvs/csvs_from_random_split/
The only warning in the logs, prior to the run is: datastore.as_mount" is deprecated after version 1.0.69. Please use "file_dataset.as_mount
Snippet of code more or less:
from azureml.core import RunConfiguration
from azureml.core import Workspace, Datastore, Experiment
from azureml.data import OutputFileDatasetConfig
ws = Workspace.from_config(args.workspace_config)
datastore = Datastore(ws, args.datastore)
root_dir_ref = datastore.as_mount()
output_dir_ref = OutputFileDatasetConfig(destination=(datastore, output_path))
if not os.path.exists(dataset_store): #dataset_store is a path in output_dir_ref
os.makedirs(dataset_store)
