Stack Overflow
Stack Overflow

Questions tagged [databricks-repos]

For questions around Databricks Repos for Git integration.

This tag is for questions around Databricks Repos - development best practices, code organization, integration with CI/CD, deployment, etc.

58 questions
1
vote
2 answers

azure databricks notebook rebase

I am trying to rebase my feature branch to the latest commit of the master branch. In the documentation I saw there is a button for rebase, but in my notebook I only see the create branch button. Any idea how to do rebase in azure databricks…
Youshikyou
  • 365
  • 1
  • 8
1
vote
1 answer

Azure databricks notebook missing Repo

I am a beginner in azure databricks notebook. I read the docs that in the azure databricks notebook, there should be a Repo in the sidebar. But in one of my notebooks, I didn't find it. Do you know why? Is it because of some setting on purpose?
Youshikyou
  • 365
  • 1
  • 8
1
vote
0 answers

Files missing from Repos in Databricks

I have an issue with files disappearing from Databricks Repos. The files exist in the branch in git locally and in the git web platform I have pulled the latest version of the branch into Databricks Repos. Yet some of the files do not exist in the…
Jorge
  • 392
  • 3
  • 14
1
vote
1 answer

Databricks API call fails on Azure DevOps pipelines using python script, but run successfully on Postman from local machine

In Azure databricks API I am trying to pull latest changes to main branch in each folder in Databricks repos by using Azure Databricks API. This is where I am refering to: When I use postman to make the calls by posting requests to the following…
E. Erfan
  • 1,239
  • 19
  • 37
1
vote
2 answers

Update Databricks Workspace Repo by Connecting to Databricks CLI with Github Actions

I'm attempting to automatically pull the latest version of a GitHub repo into my Databricks workspace every time a new push is made to the repo. Everything works fine until the Databricks CLI requests the host URL after which it fails with "Error:…
Nick_89
  • 117
  • 2
  • 11
1
vote
1 answer

Is it possible to set only one branch at Databricks shared git folder(highlighted in screenshot)?

I would like to set only one branch at shared folder in databricks workspace. Attaching screenshot to give more clarity on the same. All of data factory pipelines are using shared folder location for running notebooks and if some one changes it to…
rpshgupta
  • 135
  • 1
  • 8
1
vote
0 answers

The changes were made on Databricks repos not displaying in destination branch after merge with main

My work flow is Developer creates a feature branch from main in Databricks repos -> after they make changes on it -> they raise a pull request for merge into main in azure devops-> it triggers the CICD pipeline push the code into the destination…
Harish334363
  • 65
  • 10
1
vote
1 answer

Import a CSV file using Databricks CLI in Repos

We are using Databricks to generate ETL scripts. One step requires us to upload small csvs into a Repos folder. I can do this manually using the import window in the Repos GUI. However, i would like to do this programmatically using the databricks…
Jongen87
  • 13
  • 2
1
vote
1 answer

How to automate the bitbucket repository to databricks repos by using ci/cd pipeline

Can someone help on this how to automate the ci/cd pipeline to update and create new files in databricks repos from bitbucket repositories..
Sree
  • 19
  • 4
1
vote
1 answer

Databricks repos - unable to use dbutils.notebook.run with absolute path

I'm unable to get an absolute path working with dbutils.notebook.run(). Using the absolute path with dbutils.fs.ls(with "file:/Workspace/Repos/user_email/Datalake/databricks/models/notebook") returns a FileInfo object. Running the same filepath…
Eric Muijs
  • 29
  • 6
1
vote
1 answer

Transfer files saved in filestore to either the workspace or to a repo

I built a machine learning model: lr = LinearRegression() lr.fit(X_train, y_train) which I can save to the filestore by: filename = "/dbfs/FileStore/lr_model.pkl" with open(filename, 'wb') as f: pickle.dump(lr, f) Ideally, I wanted to save the…
Mikee
  • 783
  • 1
  • 6
  • 18
1
vote
1 answer

Running another notebook inside a notebook which is connected to git

In our company we learned to connect our notebook to git repository, however there is an issue. Running a notebook inside another notebook. This is the way that we have discovered, however this solution cant go into production because it included an…
Ali Saberi
  • 864
  • 1
  • 10
  • 33
1
vote
1 answer

Deploy repository to new databricks workspace

I am developing my code in a databricks workspace. Using integration with Repos, I use Azure DevOps to version control my code. I would like to use Azure Pipelines to deploy my code to a new test/production environment. To copy the files to the new…
Larsq
  • 315
  • 2
  • 12
1
vote
2 answers

New databricks feature: import is not working

I have uploaded my databricks notebooks to a repo and replace %run sentences with import using the new databrick public available features (Repo integration and python import):…
Nasgar
  • 859
  • 2
  • 11
  • 26
1
vote
1 answer

Databricks GitHub and Bitbucket integrations, credential conflict

I'm migrating my files from Bitbucket to Github, but every time that I need to clone something from bitbucket and send it to GitHub, I need to create a new token to integrate the tools. It seems that, on Databricks, when you save a Github…
Evandro Lippert
  • 336
  • 2
  • 11
1 2
3
4