2

TL;DR: Is it possible to have terraform perform the Microsoft Graph steps for SCIM provisioning for Azure AD and Databricks?

I am trying to configure automatic SCIM provisioning for Databricks so that user and group management can be done via Azure Active Directory instead of within Databricks itself. I am trying to figure out how to accomplish this provisioning via terraform.

I see the following Azure databricks document here for configuring SCIM provisioning manually. Looks like it can be done with an enterprise application or using Microsoft Graph.

At the time of writing there is an open pull request on the azuread terraform provider to add provisioning via what looks to be the enterprise application method. Since I don't want to wait for that PR to get merged, I've been trying to figure out if the Microsoft Graph method is achievable via terraform now. If it is, I can't figure out how to do it.

I am using the latest version of the azuread provider which says that as of 2.0 it "exclusively uses Microsoft Graph to connect to Azure Active Directory."

I've been looking at the azuread_application documentation, following the manual config steps and trying to figure out what the terraform equivalent of those steps would be but it's not clear in the first place if this is even possible.

CJA
  • 147
  • 1
  • 9

1 Answers1

1

Yes, it's completely possible to provision users from AAD to the Databricks. The implementation is relatively big to put it into the answer completely, but you can find working version here (I know that links aren't recommended, but it's > 100 lines of terraform code, plus code is updated from time to time to handle newer versions of terraform & providers). The high-level workflow is as following:

  • Create each group in Databricks using databricks_group resource & give them necessary entitlements (can create cluster, can access DBSQL, etc.)
  • For each of the specified groups fetch list of the users from AAD
  • Create a set from users of all groups - to make sure we don't have duplicates
  • Separate users from service principals and create them in Databricks using databricks_user resource (note the force flag, so it will automatically import users that were added manually)
  • Do the same for service principals using the databricks_service_principal resource
  • Assign users & service principals to the groups using the databricks_group_member resource (this is the part most complex for understanding as we need to have two nested loops)

Current limitations:

  • Doesn't handle nested groups, although this could be added
Alex Ott
  • 80,552
  • 8
  • 87
  • 132