1

From the OutputFileDatasetConfig documentation for the destination class member,

If set to None, we will copy the output to the workspaceblobstore datastore, under the path /dataset/{run-id}/{output-name}

Given I have the handle to such OutputFileDatasetConfig with destination set to None, how can I get the generated destination without recomputing the default myself as this can be subject to change.

Francois
  • 852
  • 6
  • 17

2 Answers2

0

If you do not want to pass a name and path, then in that scenario the run details should provide the run id and the path can be created using the same. In an ideal scenario you would like to pass these details, if they are not passed the recommended approach is to use them in a intermediate step so the SDK can handle this for you, as seen in this PythonScriptStep()

from azureml.data import OutputFileDatasetConfig
dataprep_output = OutputFileDatasetConfig()
input_dataset = Dataset.get_by_name(workspace, 'raw_data')

dataprep_step = PythonScriptStep(
    name="prep_data",
    script_name="dataprep.py",
    compute_target=cluster,
    arguments=[input_dataset.as_named_input('raw_data').as_mount(), dataprep_output]
    )
RohitMungi
  • 124
  • 6
0
output = OutputFileDatasetConfig()
src = ScriptRunConfig(source_directory=path,
                  script='script.py',
                  compute_target=ct,
                  environment=env, 
                  arguments = ["--output_path", output])
                  
run = exp.submit(src, tags=tags)

###############INSIDE script.py

mount_point = os.path.dirname(args.output_path)
os.makedirs(mount_point, exist_ok=True)
print("mount_point : " + mount_point)
Manuel J. Reyes Gomez
  • 1
  • Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Oct 26 '21 at 19:21