0

I have a dataframe where i need to add a column from the widget value that is being passed. I am trying the below code but its not helping in anyways. When we display(pdf) we should also see the ID column has also been added.

  pdf=pTF.withColumn('ID',lit(dbutils.widgets.text("ID", "eiheifhj", label="pro_ID")).cast(StringType()))

The above code is puhsing all nulls into the columns ID. Could you please help. I would request the experts to help in the above challenge

batman_special
  • 115
  • 1
  • 2
  • 10

1 Answers1

1

In the following code, you are actually defining the widget instead of fetching the value the widget contains.

dbutils.widgets.text("ID", "eiheifhj", label="pro_ID")
  • "ID" is the name of your widget.
  • "eiheifhj" is the default value of the widget.
  • "pro_ID" is the widget label in UI

so if you want to fetch the value of the 'ID' widget, you can use the following method.

dbutils.widgets.get(<widget_name>)

hence your program should be modified as follows :

pdf=pTF.withColumn('ID',lit(dbutils.widgets.get("ID")).cast(StringType()))
Anand Vidvat
  • 977
  • 7
  • 20