0

I have GIT integrated in Azure Databricks and want to be able to have an updated notebook in my workspace every time I do a pull request into Dev for example, thereby having any update in GIT also appear in my workspace notebook. I believe there's a way to do this via kicking off a release pipeline in DevOps when this happens, as well as possibly an API being available to help with this but I'm not finding any information.

Any information or links to articles for performing this would be very helpful, thanks.

Alex Ott
  • 80,552
  • 8
  • 87
  • 132
Rchee
  • 71
  • 1
  • 10

1 Answers1

1

That's possible to automate only when you have your notebooks in the Databricks Repos - it's not possible to update notebook in the workspace via API. When you have repository checked out, you may update it to the latest state of selected branch using the Update command of Repos REST API or using the repos update command of the Databricks CLI. For pull requests you need to figure out how to get branch name as it heavily dependent on the specific Git implementation - you can find that information in Azure DevOps documentation.

I have a working example of integrating Databricks Repos with Azure DevOps for testing of code in the notebooks and promotion between stages using Databricks CLI. Repository contains detailed instructions on how to setup everything - it's too much to put everything here.

Alex Ott
  • 80,552
  • 8
  • 87
  • 132
  • Thanks for this! I have notebooks in the repo that were previously in the workspace only, however, because we want ADF to call these notebooks we still need them in the updated workspace – Rchee Jun 02 '22 at 15:23
  • 1
    ADF works fine with Databricks Repos - it's just another top-level folder – Alex Ott Jun 02 '22 at 15:34