0

I need to automate task which is related to azure databricks.we have configured a job in azure databricks and suddenly my service prinicipal secrets get expired and my notebook failed. on the next day I created new secrets and unmount --> mount it again and job worked. I know the way to create new secret or get the alert before expiring using logic app and manually will change it.

But how to manage unmount --> mount step automatically? As SP can be used in multiple project. This is how I am mounting and mount_point used across notebook. enter image description here

anuj
  • 124
  • 2
  • 13

1 Answers1

0

Here is the sample code for Unmounting and mounting notebook. Check if the path is mounted or not, If it is not mounted yet mount the path with the given condition.

enter image description here

def mount_blob_storage_from_sas(dbutils, storage_account_name, container_name, mount_path, sas_token, unmount_if_exists = True):
    if([item.mountPoint for item in dbutils.fs.mounts()].count(mount_path) > 0):
        if unmount_if_exists:
        print('Mount point already taken - unmounting: '+mount_path)
        dbutils.fs.unmount(mount_path)
    else:
        print('Mount point already taken - ignoring: '+mount_path)
        return
        print('Mounting external storage in: '+mount_path)
          dbutils.fs.mount(
            source = "wasbs://{0}@{1}.blob.core.windows.net".format(container_name, storage_account_name),
            mount_point = mount_path,
            extra_configs = {"fs.azure.sas.{0}.{1}.blob.core.windows.net".format(container_name, storage_account_name): sas_token })

Create a job to run the notebook with given specific time period.

enter image description here

Provision the notebook path and cluster to create the JOB.

enter image description here

Schedule a time to trigger the notebook in Add Schedule .

enter image description here

Select Scheduled path to provide time to trigger.

enter image description here

Job will trigger at given time to run the notebook.

enter image description here

Nagasai
  • 41
  • 3