Grant Access to Azure Databricks using Azure Devops

Question

I am fairly new to Azure Devops and Azure Databricks.

I have created Azure Databricks workspace using Azure Devops CI/CD Pipeline. Now I am looking for a solution to Add Users to Azure Databricks workspace using DevOps Pipeline.

Answer 1

There are several methods:

• Use databricks_user resource from Databricks Terraform provider - it could be as simple as example below, or you can combine it with azuread provider, and pull users from Azure Active Directory. Another advantage of Terraform provider is that you can combine it with user groups, and other things.

resource "databricks_user" "me" {
  user_name = "me@example.com"
}

• Use Databricks SCIM API for Users (part of general SCIM API) - creation is quite straightforward, you just need to form correct JSON as described in docs (I didn't want to copy JSON from there), and do a call with curl or something like that. Also,

• There is a collection of PowerShell scripts developed by the DataThirst company, that include scripts for adding & removing users, etc. These scripts are using REST API under the hood, and could be simpler than to use REST API. Some of these tasks are also available on the DevOps marketplace.

In any case, you need to authenticate to the workspace. For automated pipelines you have two choices - service principals or managed identity associated with DevOps worker, and they should have Owner or Contributor permissions on the workspace level, or be added into workspace as admin users.

• For REST API authentication of service principal is described in details in documentation, for managed identity you just get the token from internal REST API.
• Databricks Terraform provider also supports both service principals and managed identity.