Issue with mount persistent storage in workflow, File not found

Question

When I used fn.apply(mlrun.mount_v3io()) in MLRun function, it worked correctly and I can access to files in mount persistent storage.

import mlrun 
...
fn = mlrun.import_function("test-function.py")

# use mount_v3io() for iguazio volumes
fn.apply(mlrun.mount_v3io())

The issue was, when I used the fn.apply(mlrun.mount_v3io()) in MLRun workflow, I got error message about non exist file (the mount did not work):

Error 1 Metadata file
'rpd.pkl' could not be found v3io://protected/iron/

Did you solve the similar issue in MLRun workflow?

score 0 · Accepted Answer · answered May 10 '23 at 17:02

In case of MLRun workflow, you have to use init_functions

# init functions is used to configure function resources and local settings
def init_functions(functions: dict, project=None, secrets=None):
    for f in functions.values():
        f.apply(auto_mount())

def kfpipeline():
    taxi_records_csv_path = mlrun.get_sample_path('data/Taxi/yellow_tripdata_2019-01_subset.csv')
    zones_csv_path = mlrun.get_sample_path('data/Taxi/taxi_zones.csv')
    
    # build our ingestion function (container image)
    builder = funcs['taxi'].deploy_step(skip_deployed=True)
    ...

You can see full sample here see title Create the Workflow

Issue with mount persistent storage in workflow, File not found

1 Answers1