Azure DevOps Release pipeline for Databricks notebook

Question

I am creating a Azure DevOps Release Pipeline for Databricks notebooks deployment.

I have written below code in powershell activity but its not working.

Requesting some pointer here.

# Variables
$workspaceUrl = "https://<databricks-instance>"
$personalAccessToken = "<databricks-personal-access-token>"
$notebookPath = "/notebooks/my_notebook.ipynb"
$targetEnvironment = "UAT"

# Authenticate with Databricks CLI
databricks configure --token
databricks configure --token "$personalAccessToken" --url "$workspaceUrl"

# Deploy Notebook to UAT Environment
databricks workspace import -l "$notebookPath" -o "/$targetEnvironment$notebookPath"

What does "not working" mean? Please be explicit with the issues you're encountering when asking for help. — Daniel Mann, Jul 02 '23 at 14:54

score 0 · Answer 1 · answered Jul 02 '23 at 09:19

You don't need to configure CLI explicitly, it's just enough to expose workspace URL as DATABRICKS_HOST environment variable, and personal access token as DATABRICKS_TOKEN environment variable - then CLI will automatically pickup them. See documentation for full list of supported variables.

It's also not recommended to specify PAT directly in the code, but instead it's better to put it & workspace URL into variables and refer from the pipeline, like this (full example):

    - script: |
        echo "Checking out the $(branchName) branch"
        databricks repos update --path $(STAGING_DIRECTORY) --branch "$(branchName)"
      env:
        DATABRICKS_HOST: $(DATABRICKS_HOST)
        DATABRICKS_TOKEN: $(DATABRICKS_TOKEN)
      displayName: 'Update Staging project'

Azure DevOps Release pipeline for Databricks notebook

1 Answers1