MLFlow how to change backend store uri from file storage to DB

Question

I am using mlflow tracking with file storage as backend store for a while, I have a lot of runs logged in the system.
Lately I wanted to start using the model registry but unfortunately this feature is currently supported only with DB as the backend store.
How can I change the backend store without loosing all the runs that I have already logged?

The command that I am using to run the server:

mlflow server --backend-store-uri /storage/mlflow/runs/ --default-artifact-root /storage/mlflow/artifactory/ --host 0.0.0.0 --port 5000

I have been looking for the same feature, and after a long read of the documentation, I do not think it's available "out of the box". We would need to implement this manually by reading the old store, then logging to the new store. Of course, automatic metadata (date, commit, etc.) would not be correct. — PhilMacKay, Dec 16 '20 at 16:54

Garvit · Answer 1 · 2022-01-24T05:44:41.523

It's true that we need a database if want to use the model registry feature. This is how I set up (using MySQL) on my Linux machine in just a few steps:-

1)- Install MySQL in your system.

sudo apt install mysql-server

2)- Create a database to use as an MLflow backend tracking server.

CREATE DATABASE mlflow_tracking_database;

3)- Start MLflow tracking server using MySQL as a backend tracking store.

 mlflow server \
   --backend-store-uri  mysql+pymysql://root@localhost/mlflow_tracking_database \ 
   --default-artifact-root  file:/./mlruns \
   -h 0.0.0.0 -p 5000

4)- Set the MLflow tracking uri (within code section).

mlflow.set_tracking_uri("http://localhost:5000")

NOTE: In the 3rd step, the command automatically creates all the necessary tables within the database and uses MySQL as a backend store instead of a local file system.

score 0 · Answer 2 · edited Oct 16 '21 at 09:27

My workaround is creating service with expected working dir:

Description=MLflow Tracking Server
Wants=network-online.target
After=network-online.target

[Service]
Restart=on-failure
RestartSec=30
StandardOutput=file:/var/log/mlflow/mlflow.log
StandardError=file:/var/log/mlflow/error.log
User=root

ExecStart=/usr/local/bin/mlflow server --host 0.0.0.0 --port 5000 --backend-store-uri sqlite:///mlflow.db --default-artifact-root /drl/artifacts
WorkingDirectory=/drl

[Install]
WantedBy=multi-user.target

score -1 · Answer 3 · edited Jul 17 '20 at 21:06

-1

mlflow server --backend-store-uri sqlite:///mlflow.db --default-artifact-root wasbs://<azure_blob_container_name>@<azure_blob_account_name>.blob.core.windows.net --host 0.0.0.0

this way, the sqlite will be automatically created

edited Jul 17 '20 at 21:06

Dharman

30,962
25
85
135

answered Jul 17 '20 at 04:33

Xiaowu Hu

1

MLFlow how to change backend store uri from file storage to DB

3 Answers3