I am using Azure DevOps and Databricks. I created a simplified CI/CD Pipeline which triggers the following Python script:
existing_cluster_id = 'XXX'
notebook_path = './'
repo_path = '/Repos/abc@def.at/DevOpsProject'
git_url = 'https://dev.azure.com/XXX/DDD/'
import json
import time
from datetime import datetime
from databricks_cli.configure.config import _get_api_client
from databricks_cli.configure.provider import EnvironmentVariableConfigProvider
from databricks_cli.sdk import JobsService, ReposService
config = EnvironmentVariableConfigProvider().get_config()
api_client = _get_api_client(config, command_name="cicdtemplates-")
repos_service = ReposService(api_client)
repo = repos_service.create_repo(url=git_url, provider="azureDevOpsServices", path=repo_path+"_new")
When I run the pipeline I always get an error (from the last line):
2022-12-07T23:09:23.5318746Z raise requests.exceptions.HTTPError(message, response=e.response)
2022-12-07T23:09:23.5320017Z requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://adb-XXX.azuredatabricks.net/api/2.0/repos
2022-12-07T23:09:23.5321095Z Response from server:
2022-12-07T23:09:23.5321811Z { 'error_code': 'BAD_REQUEST',
2022-12-07T23:09:23.5322485Z 'message': 'Remote repo not found. Please ensure that:\n'
2022-12-07T23:09:23.5323156Z '1. Your remote Git repo URL is valid.\n'
2022-12-07T23:09:23.5323853Z '2. Your personal access token or app password has the correct '
2022-12-07T23:09:23.5324513Z 'repo access.'}
In Databricks, I connect my repo with Azure DevOps: In Git I created a full access token which I added to Databricks' Git Integration and I am able to pull and push in Databricks.
For my CI/CD pipeline, I created variables containing my Databricks Host address and my token. When I change the token, I get a different error message (403 http code) - so the token seems to be fine.
Here a screenshot of my variables.
I have really no clue what I am doing wrong. I tried to run a simplified version of the official Databricks code here.
