2

I'm using the new Databricks Repos functionality and in Data Factory UI for the notebook activity you can browse the Databricks workspace and select Repos > username > project > folder > notebook.

Is it possible to call a Databricks notebook in a specific branch from Data Factory?

Alex Ott
  • 80,552
  • 8
  • 87
  • 132
baatchen
  • 469
  • 1
  • 7
  • 16

2 Answers2

1

You can (now) call only the notebook that belongs to checkout of specific version - if you have checkout in another version, then it's impossible to call another version. Like, if you have checkout at master but want to call code from release branch, then you need to have checkout of that branch.

But you may have multiple Repos associated with the same Git repository, but with different versions checked out.

Alex Ott
  • 80,552
  • 8
  • 87
  • 132
0

You can't call a specific branch in Databricks from Data Factory. Our solution is creating multiple folders in Databricks with the same repository but a different branch.

With dynamic content in Data Factory we can load the paths from SQL Server and call them. (or hardcode it with if/else and vars in DF).

different branches in databricks

54m
  • 719
  • 2
  • 7
  • 18