Is there a function similar to awsglue's getResolvedOptions that will work in an azure databricks notebook using python?

Question

I'm moving code into azure and am wondering what function would be used to accomplish the same thing in python, but without awsglue:

import sys
from awsglue.utils import getResolvedOptions
args = getResolvedOptions(sys.argv, ['JOB_NAME', 'day_partition_key'])
print(args['day_partition_key'])

score 0 · Answer 1 · answered Mar 27 '21 at 19:07

In Databricks, if you're using Notebook for data processing, then you can pass parameters to a job when creating it as described in documentation (step 6), and then access these parameters via widgets, for example, if you need to pass parameter, then you declare it as widget with name param1:

dbutils.widgets.text("param1", "default_value", "Description")

and extract the value with:

param1 = dbutils.widgets.get("param1")

and declare the same parameter when creating the job.

If you're using Python or Jar job, then you receive parameters as usual, from command-line.

Is there a function similar to awsglue's getResolvedOptions that will work in an azure databricks notebook using python?

1 Answers1