1

Is there a way to persist a Python machine learning model when using the free Databricks community edition?

  • It looks like the DBFS is not available. This means that I can't use tools like joblib to save the model in the file system.
  • ML Flow is not available in community edition, so that's not an option either.

Is there some other way I'm missing?

desertnaut
  • 57,590
  • 26
  • 140
  • 166
Trey
  • 201
  • 3
  • 14

1 Answers1

0

You can always persist the model to the local disk, like, /tmp/my-model, and then use dbutils.fs.cp function to copy it to DBFS to persist, like this:

dbutils.fs.cp('file:/tmp/my-model', '/FileStore/my-model')

When you will need it again, just copy in the reverse direction & load:

dbutils.fs.cp('/FileStore/my-model', 'file:/tmp/my-model')
Alex Ott
  • 80,552
  • 8
  • 87
  • 132