0

I'm trying to use the mlflow databricks integration, specifically the tracking API. Normally, I can view past runs info in the handy sidebar of a notebook, as you can see here and which I got from the tutorial. However, what I want now is to use multiple notebooks to send runs to the same experiment. Additionally, I would like to view the results of all these common runs in each of the notebooks. To do this, I need to change the (default) experiment tracked by the "runs" tab.

Ultimately, my question boils down to the following: how can I set the experiment being tracked by the "runs" tab? I have tried using mlflow.set_tracking_uri and mlflow.set_experiment(mlflow_experiment_name)

information_interchange
  • 2,538
  • 6
  • 31
  • 49

1 Answers1

1

I don't believe this is possible today, as the design choice is to associate the runs tab to the notebook experiment. From the docs:

Every Python and R notebook in a Databricks workspace has its own experiment. When you use MLflow in a notebook, it records runs in the notebook experiment.

A notebook experiment shares the same name and ID as its corresponding notebook. The notebook ID is the numerical identifier at the end of a Notebook URL.

You can create experiments independent of the notebook experiment and log runs to it from different sources. You'll still have to open up the tracking UI to explore the results though.

In other words, you can send multiple runs from different notebooks to the same experiment, but today you cannot log multiple runs to the 'Runs' tab in a specific notebook.

Community
  • 1
  • 1
Raphael K
  • 2,265
  • 1
  • 16
  • 23