Stack Overflow
Stack Overflow

Questions tagged [user-defined-functions]

A function provided by the user of a program or an environment most often for spreadsheet type applications or database applications. Use [custom-functions-excel] for Excel and [custom-function] for Google sheets. Specify a programming language tag as well: [google-apps-script], [javascript], [sql], [tsql], etc. as well as a tag for the application: [excel], [google-spreadsheet], [sql-server] etc.

In the context of a programming language or an environment, a User Defined Function (UDF) is a function that is created by a user to perform a specific task (as opposed to a function that is intrinsic to the environment, built into the programming language or environment).

Spreadsheet applications like Excel and Google Sheets calls these "custom functions".

Microsoft also uses the term User Defined Functions with . The tag may also be applicable. See What is the need for user-defined functions in SQL Server?

Use:

4875 questions
1
vote
1 answer

pyspark: StopWordsRemover with user defined functions (UDF)

I have a dataframe consisting of text and languages sf = spark.createDataFrame([ ('eng', "I saw the red balloon"), ('eng', 'She was drinking tea from a black mug'), ('ger','Er ging heute sehr weit'), ('ger','Ich habe dich seit…
Rory
  • 471
  • 2
  • 11
1
vote
2 answers

Python function make parameter mandatory if another parameter is passed

I have a function in Python as def myfunc(a, b=None, c=None): my code here Is there a way to ensure that if the user passes the value of the parameter b it is necessary to pass the parameter c to myfunc?
S_S
  • 1,276
  • 4
  • 24
  • 47
1
vote
1 answer

pyspark - Index Error when applying UDF over Sliding Window

I am working with a pyspark data frame with users, dates and locations. My goal is to implement a sliding window of 3 days [-1 days, 1 days] and calculate the most common location inside the window. +---+-------------+----+------------+---------+ |…
s223
  • 59
  • 6
1
vote
1 answer

UDF (user-defined function) in Bigquery

I want to use SQL query in udf and get some column value from it and have some calculation through it. I am not able to write SQL inside permanent udf function. I want something like that -> create function dataset.fun(j int64) language js as…
D7-PO
  • 21
  • 4
1
vote
2 answers

Proper procedure syntax for Postgres Function as Procedure in BigQuery?

I have the following function in postgres: create function my_function(price numeric, qty numeric, min_charge numeric, other_fee numeric) returns numeric language plpgsql as $$ DECLARE _charge numeric; BEGIN IF qty = 0 THEN …
Mark McGown
  • 975
  • 1
  • 10
  • 26
1
vote
2 answers

How to create UDF in BigQuery? Routine name missing dataset

The following is the definition of a working function I have in a postgres db: create function my_function(price numeric, qty numeric, min_charge numeric, other_fee numeric) returns numeric language plpgsql as $$ DECLARE _charge…
Mark McGown
  • 975
  • 1
  • 10
  • 26
1
vote
2 answers

Use parameter in UDF or Stored Procedure to return table

I have a case when using the Snowflake API. We have a bunch of materialized views, multiple for each customer of ours. When using the API, we would like to use functions or stored procedures to produce a result in runtime, and taking the customer ID…
Kristoffer Andersson
  • 11
  • 1
1
vote
1 answer

How to convert Scalar Pyspark UDF to Pandas UDF?

I have a UDF as below which is a normal scalar Pyspark UDF : @udf() def redact(colVal: column, offset: int = 0): if not colVal or not offset: return 'X'*8 else: charList=list(colVal) …
ASHISH M.G
  • 522
  • 2
  • 7
  • 23
1
vote
1 answer

How to iterate through column name of a table and pass value to UDF in MSSQL while loop

TableName: Stocks I am trying to fetch profit or loss of stock company in stocks table.(Refer output table in below screenshot) I had created User defined function with passing parameter as stock company and return integer value which shows wither…
PavanKumar GVVS
  • 859
  • 14
  • 45
1
vote
1 answer

UDFs Use Array to Split String in Column

I have a df that has two columns. One column is a string and the other is an array of integers. root |-- col1: string (nullable = true) |-- col2: array (nullable = true) | |-- element: integer (containsNull = true) The dataframe looks…
dcrowley01
  • 141
  • 2
  • 12
1
vote
1 answer

Finding the frequency of an element in an array using a user-defined function in C

I am writing a program where we need to find the frequency of the number chosen in an array. The numbers were inputted by the user and he/she will choose a number to count its instance. I did it with this program. #include int main() { …
Dexter1403
  • 35
  • 6
1
vote
1 answer

pandas_udf to extract a value from a column containing maps

I have the following spark df id | country ------------------ 1 | Null 2 | {"date": null, "value": "BRA", "context": "nationality", "state": null} 3 | {"date": null, "value": "ITA", "context": "residence", "state": null} 4 | {"date": null,…
Tytire Recubans
  • 967
  • 10
  • 27
1
vote
0 answers

Creating a UDF in pyspark to select a column and parsing each row through beautiful soup to get a string

I have a python pyspark block of code that collects data from a dataframe column(Body) and I am able to use beautifulsoup to parse the

tags paragraph for each row and turn it to a long string. text_list = [] for row in…

Jay Jay
  • 33
  • 8
1
vote
2 answers

Decompress base64 encoded, gzipped JSON

I've got base64 encoded, gzipped json in a Snowflake binary column that I'm trying to unpack. With the query: select base64_encode(my_binary_data) as my_base64_string from my_table I get a base64 encoded string, which I can cut n' paste into a…
NoobsterNoob
  • 125
  • 1
  • 2
  • 7
1
vote
1 answer

Adding the result of a function in a Dataframe column [Spark Scala]

I want to do some calculations and add that to an existing dataframe. I have the following function to calculate the address space based on the longitude and lattitude. def getH3Address(x: Double, y: Double): String ={ …
EnesK
  • 93
  • 6
1 2 3
…
99 100