Azure Databricks notebook not picking the parameters passed via job api

Question

Flask application --> Capturing the value entered via web form and passing it as a parameter to trigger a databricks job.

Tried passing the parameters as follows :

1- payload = {
>             'notebook_params': {
>                 'customer_name': customer_name
>             }
2- payload = {
>             "notebook_task": {
>             "notebook_path": "/Shared/.../",
>             "base_parameters": {           
>                 "customer_name": customer_name,
>                 }
>             }
>         }

While creating the job, passed the default parameter but this does not gets overridden, if I removed it than could see blank value is getting passed. enter image description here

In Databricks Notebook, the print command returns blank if no default value is set.

# Get the customer_name value from the parameter
dbutils.widgets.get("customer_name", "","")
customer_name = dbutils.widgets.get("customer_name")

# Use the customer_name in your notebook logic
print("Customer Name:", customer_name)

Could see the following Run parameters are used to execute the job enter image description here

I need assistance to identify if I am doing anything wrong and how I can overcome the issue.

# Get the customer_name value from the parameter
dbutils.widgets.get("customer_name", "", "")
customer_name = dbutils.widgets.get("customer_name")

# Use the customer_name in your notebook logic
print("Customer Name:", customer_name)

The above should return the customer_name= Cx4 that was passed as parameter when the job run was created via api call from the flask app.

Answer 1

For such situation , you should try executing the Job run via a rest api call from Postman. That will tell you if the issue is with your flask app or your databricks notebook.

Try with the following databricks code in your notebook. I'm defining that the input is a string/text first before getting it. It should work.

# Get the customer_name value from the parameter
dbutils.widgets.text("customer_name","")
customer_name = dbutils.widgets.get("customer_name")

# Use the customer_name in your notebook logic
print("Customer Name:", customer_name)

Answer 2

You also need to specify the parameter here: https://i.stack.imgur.com/e70Q3.png