3

My goal is to configure ML Flow to run on a remote Linux server, with logs stored in the PostgreSQL database and artifacts in /home/aw/mlfow/mllogs, where "aw" is my user name with root privileges.

When I run my simple python code (see section below):

  1. when it's last line ("mlflow.log_artifact("features.txt")) is commented out: I get no errors in Python, but when I enter log details via browser, in the artifact section I get the warning: Unable to list artifacts stored undersftp://mlflow_user@194.39.141.27:~/mlflow/mlruns/0/d1eb9ce83b6b4ede96a9ea5203c097da/artifacts

  2. in case the last line is active, python compiler returns a long list of errors, ending with ValueError: Port could not be cast to integer value as '~' I tried running the server from the CLI in many different ways, each time changing the --default-artifact-root parameter (see section below). Interestingly, the printout of the mlflow.get_artifact_uri() variable from the Python code remains the same, despite changes in the CLI server parameters: it always shows up as sftp://mlflow_user@94.39.141.27:~/mlflow/mlruns/0/c613c110839946a3adc198377cc82c0c/artifacts

So, it looks like there is a problem of setting this parameter up during MLFlow server run from CLI. Maybe it's cached somewhere? Linux server reboot doesn't help.

Code to reproduce issue Server CLI run (credentials are just examples):

option 1 (based on https://towardsdatascience.com/setup-mlflow-in-production-d72aecde7fef): mlflow server --backend-store-uri postgresql://mlflow_user:mlflow321@localhost/mlflow_db --default-artifact-root sftp://mlflow_user@194.39.141.27:~/mlflow/mlruns -h 0.0.0.0 -p 8000&

options 2.1 and 2.2 (aw is my user name on the machine, with root privileges): mlflow server --backend-store-uri postgresql://mlflow_user:mlflow321@localhost/mlflow_db --default-artifact-root sftp://aw:aw_pass@194.39.141.27:~/mlflow/mlruns -h 0.0.0.0 -p 8000&

mlflow server --backend-store-uri postgresql://mlflow_user:mlflow321@localhost/mlflow_db --default-artifact-root sftp://aw:@194.39.141.27:~/home/aw/mlflow/mlruns -h 0.0.0.0 -p 8000&

option 3: mlflow server --backend-store-uri postgresql://mlflow_user:mlflow321@localhost/mlflow_db --default-artifact-root sftp://mlflow_user:mlflow_pass#@194.39.141.27:~/mlflow/mlruns -h 0.0.0.0 -p 8000&

Python code:

import mlflow

if __name__ == "__main__":
    mlflow.set_tracking_uri("http://194.39.141.27:8000") #hostname IP here is just an example

    features = "rooms, zipcode, median_price, school_rating, transport"
    with open("features.txt", 'w') as f:
        f.write(features)

    with mlflow.start_run():

        tracking_uri = mlflow.get_tracking_uri()
        artifact_uri = mlflow.get_artifact_uri()
        print("Tracking uri: {}".format(tracking_uri))
        print("Artifact uri: {}".format(artifact_uri))

        mlflow.log_artifact("features.txt")

Other info / logs Python compiler log:

File "1.py", line 18, in
mlflow.log_artifact("features.txt")
File "/home/aw/anaconda3/lib/python3.8/site-packages/mlflow/tracking/fluent.py", line 544, in log_artifact
MlflowClient().log_artifact(run_id, local_path, artifact_path)
File "/home/aw/anaconda3/lib/python3.8/site-packages/mlflow/tracking/client.py", line 903, in log_artifact
self._tracking_client.log_artifact(run_id, local_path, artifact_path)
File "/home/aw/anaconda3/lib/python3.8/site-packages/mlflow/tracking/_tracking_service/client.py", line 271, in log_artifact
artifact_repo = self._get_artifact_repo(run_id)
File "/home/aw/anaconda3/lib/python3.8/site-packages/mlflow/tracking/_tracking_service/client.py", line 262, in _get_artifact_repo
return get_artifact_repository(artifact_uri)
File "/home/aw/anaconda3/lib/python3.8/site-packages/mlflow/store/artifact/artifact_repository_registry.py", line 102, in get_artifact_repository
return _artifact_repository_registry.get_artifact_repository(artifact_uri)
File "/home/aw/anaconda3/lib/python3.8/site-packages/mlflow/store/artifact/artifact_repository_registry.py", line 71, in get_artifact_repository
return repository(artifact_uri)
File "/home/aw/anaconda3/lib/python3.8/site-packages/mlflow/store/artifact/sftp_artifact_repo.py", line 32, in init
"port": parsed.port,
File "/home/aw/anaconda3/lib/python3.8/urllib/parse.py", line 174, in port
raise ValueError(message) from None
ValueError: Port could not be cast to integer value as '~'

To conclude: My main goal is to store MLFLow artifacts on a remote Linux server. I would really appreciate any help on how to do this?

Progman
  • 16,827
  • 6
  • 33
  • 48
Andrzej Wodecki
  • 107
  • 1
  • 8

0 Answers0