Stack Overflow
Stack Overflow

Questions tagged [udf]

A user-defined function (UDF) is a function provided by the user of a program or environment, in a context where the usual assumption is that functions are built into the program or environment. Although the term is widely known in Hadoop components such Hive and Pig, it is also used in other contexts such programming languages and some DBMSs.

From the docs:

Introduction

Pig provides extensive support for user defined functions (UDFs) as a way to specify custom processing. Pig UDFs can currently be implemented in three languages: Java, Python, and JavaScript.

The most extensive support is provided for Java functions. You can customize all parts of the processing including data load/store, column transformation, and aggregation. Java functions are also more efficient because they are implemented in the same language as Pig and because additional interfaces are supported such as the Algebraic Interface and the Accumulator Interface.

Limited support is provided for Python and JavaScript functions. These functions are new, still evolving, additions to the system. Currently only the basic interface is supported; load/store functions are not supported. Furthermore, JavaScript is provided as an experimental feature because it did not go through the same amount of testing as Java or Python. At runtime note that Pig will automatically detect the usage of a scripting UDF in the Pig script and will automatically ship the corresponding scripting jar, either Jython or Rhino, to the backend.

537 questions
0
votes
1 answer

How to call a UDF in master database on a linked server

I am unable to call a UDF in a Linked Server. Here's my T-SQL code: DECLARE @Phone_Digits_Input varchar(12); DECLARE @Phone_Min_Max_Input varchar(5); DECLARE @Phone_Digits_String varchar(10); DECLARE @Phone_Min_Max_String varchar(5); DECLARE…
Piyush Varma
  • 27
  • 5
0
votes
1 answer

Simple Python UDF issue for Hadoop pig

I write a very simple Python and here is my UDF code, pig code and error message, any ideas what is wrong? Thanks. UDF (test.py), @outputSchema("cookie:chararray") def getSimple(): return 'Hello' Pig code, register test.py using jython as…
Lin Ma
  • 9,739
  • 32
  • 105
  • 175
0
votes
2 answers

Using a UDF to populate a series of 1's and 0's based on string

I have a table in sql that has a column named [message]. I need to use a UDF to populate a 1 or a 0 dependant on if that substring contains 'LHL'. My code: create function dbo.generate ( @result varchar(max) ) returns int …
Ashley Krynen
  • 31
  • 6
0
votes
1 answer

java udf for adding columns

i am writing java udf function to add the pincode by comparing the locality column.here is my code. import java.io.IOException; import org.apache.pig.EvalFunc; import org.apache.pig.data.Tuple; import org.apache.commons.lang3.StringUtils; …
sharon paul
  • 93
  • 2
  • 9
0
votes
2 answers

#VALUE error when copying sheets

I´m using a UDF that is basically a vlookup simplified. Here´s the code: Function SUELDOBASICO(Columna As Integer) As Double SUELDOBASICO = Application.WorksheetFunction.VLookup(Application.Caller.Parent.Cells(Application.Caller.Row, 3),…
Santiago Settecasi
  • 28
  • 7
0
votes
1 answer

PIG sum corresponding elements in tuple

I have a big bag of tuples containing constant, but unknown number of integers (over 200). Is there any way to sum corresponding elements from this tuple? For example SUM_TUPLES({(1, 0, 1), (2, 1, 0)}) should return (3, 1, 1). I wrote my UDF in…
Piotr Dabkowski
  • 5,661
  • 5
  • 38
  • 47
0
votes
1 answer

Delphi UDF function to C

I'm trying to rewrite the following UDF function from Delphi to C but I don't know which parameter type should I use instead of PISC_QUAD and how to extract the month number from the param value. function GetMonthShortName(ib_date: PISC_QUAD):…
Fabrizio
  • 7,603
  • 6
  • 44
  • 104
0
votes
1 answer

While calling, which database does the UDF (User defined function) automatically refer to in Teradata

While calling a user defined function, if there is no database explicitly mentioned, which database will be considered? Is there any control setting which specifies which database will be referred to first?
Piyush Lohana
  • 43
  • 2
  • 7
0
votes
1 answer

Using Hive UDF's in Pig

Is there any reason not use Hive UDF's in Pig 0.15? I'm thinking mostly about performance, but if there are any other reasons I'd be happy to hear them. For example, we have a simple Java implementation of lpad that we use. Should we bother keeping…
Eyal
  • 3,412
  • 1
  • 44
  • 60
0
votes
0 answers

How to pass the value from one load statement into another load statement in pig script

Hi i have two load statements A and B.I want to pass the particular column values from A to B .I tried the following code. A = LOAD '/user/bangalore/part-m-00000-bangalore' using PigStorage ('\t') as…
illakya ramesh
  • 39
  • 4
0
votes
1 answer

Hive UDTF not accepting more than 2 columns output

The hive UDTF I coded, works fine as along as the number of output columns specified is two. But the moment, I change it to three and redeploy, it says the following error message. FAILED: SemanticException [Error 10083]: The number of aliases…
Rangesh Ananthalwar
  • 168
  • 1
  • 10
0
votes
1 answer

User defined function in VBA not working and returns zero, no data type mismatch

I'm defining a user defined function as follows, when I am trying to call it in a subroutine, it returns a "zero" value, which surely is wrong. Function Getpartialderiv_K_x(x As Variant, y As Variant, P As Variant, T As Variant, hx As Variant, dx…
sjj
  • 31
  • 1
  • 4
0
votes
3 answers

Hive GenericUDF return array Error

Im new with GenericUDF. I'm try to generate a function to create a telephone numbers with the use of Array. But I have an ERROR: Caused by: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazy.LazyString cannot be cast to …
ChAkO
  • 1
  • 3
0
votes
1 answer

Excel 2010 User Defined Function that returns a result BUT Also sets Cell Comments Crashes Excel when User clicks Fx

We have a User Defined VBA Function. This function returns a result, but also sets the comment of the cell. It works, we get the result and the comment is set. We have included user Help on this function. However, if the user clicks on the Fx to…
APLMom
  • 1
  • 1
0
votes
1 answer

Pig - Python UDF issue

I am trying to load a .mmdb file in Pig to then pass it into a python script. However I get the error message: "Invalid scalar projection: db: A column needs to be projected from a relation for it to be used as a scalar". My code is: REGISTER…
bjurstrs
  • 949
  • 2
  • 7
  • 17
1 2 3
…
35 36