Questions tagged [udf]

A user-defined function (UDF) is a function provided by the user of a program or environment, in a context where the usual assumption is that functions are built into the program or environment. Although the term is widely known in Hadoop components such Hive and Pig, it is also used in other contexts such programming languages and some DBMSs.

From the docs:

Introduction

Pig provides extensive support for user defined functions (UDFs) as a way to specify custom processing. Pig UDFs can currently be implemented in three languages: Java, Python, and JavaScript.

The most extensive support is provided for Java functions. You can customize all parts of the processing including data load/store, column transformation, and aggregation. Java functions are also more efficient because they are implemented in the same language as Pig and because additional interfaces are supported such as the Algebraic Interface and the Accumulator Interface.

Limited support is provided for Python and JavaScript functions. These functions are new, still evolving, additions to the system. Currently only the basic interface is supported; load/store functions are not supported. Furthermore, JavaScript is provided as an experimental feature because it did not go through the same amount of testing as Java or Python. At runtime note that Pig will automatically detect the usage of a scripting UDF in the Pig script and will automatically ship the corresponding scripting jar, either Jython or Rhino, to the backend.

537 questions
0
votes
1 answer

Passing a column to a UDF

I have a table which has a lot of SSN's and that needs to be passed to a UDF that would validate it and say if its valid or not. For example, when I execute the following query I get an error: SELECT [dbo].[f_Is_Valid_SSN_Regex]( (select SSN from…
rvphx
  • 2,324
  • 6
  • 40
  • 69
0
votes
1 answer

Apache Pig - calling Java UDF ToJSON multiple times in a script

(First post!) I’ve been playing with an example resume dataset. The resume object is somewhat complex, with multiple sub-objects. For the current phase of my plan, I’m trying to flatten the dataset by storing the sub-objects as JSON strings. I’m…
0
votes
2 answers

How To Add A JAR To Hive

I have been banging my head against the wall for a while now on how to get a hive equivalent of MS SQL's IDENTITY column added to a table and auto incremented. I have found many references to org.apache.hadoop.hive.contrib.udf.UDFRowSequence but I…
wergeld
  • 14,332
  • 8
  • 51
  • 81
0
votes
1 answer

FileNotFoundException in hive UDF

my UDF: import java.text.SimpleDateFormat; import java.util.Date; import org.apache.hadoop.hive.ql.exec.UDF; import org.apache.hadoop.io.IntWritable; public class HoursDiff extends UDF { //private = new Text(); public IntWritable…
thambisetty
  • 61
  • 1
  • 10
0
votes
0 answers

How to install MySQL UDF in WAMP Windows 10

I am trying to install https://github.com/hholzgra/mysql-udf-regexp/blob/master/README.md into WAMP on a Windows 10 64bit machine. I cannot figure it out for the life of me. Help please!
alutz
  • 192
  • 1
  • 1
  • 16
0
votes
1 answer

Pig UDF, file in Distributed Cache deleted during batch work

public class GetCountryFromIP extends EvalFunc { @Override public List getCacheFiles() { List list = new ArrayList(1); list.add("/input/pig/resources/GeoLite2-Country.mmdb#GeoLite2-Country"); return…
0
votes
2 answers

UDF to make a kind of Substitute Array-Function

I'm trying to make a function that not only substitute one text with another, but substitute a set of values from a range with another lateral set of values. I have this: Public Function SubstituteRange(RangeWithText As Range, TwoColumnMatrix As…
JoeJoe
  • 64
  • 7
0
votes
1 answer

When I call a function within a function it skips code and terminates program?

It is a big program. I stripped off unnecessary code. I left only one of the key functions When I call ss(); in any function the function gives control back to main() without accepting a string. The code works if I don't use a function to accept the…
0
votes
1 answer

How do I call a UDF in a select statement?

How do I develop a tabular UDF that does not accept any values and calculates the commission for each employee_id. I have a table called called campaign offer Create Table Campaign_Offer ( Offer_id integer identity(1,1) primary…
user2127184
  • 131
  • 1
  • 13
0
votes
1 answer

Call function for subset of rows

I have the following data in a SQL Server 2012 database: CategoryID Keyword Type ------------------------------------- 1 open plan 0 1 kitchen 0 2 air conditioned 3 2 …
newbie_86
  • 4,520
  • 17
  • 58
  • 89
0
votes
1 answer

Calling Python UDF from PIG

I have an issue while calling Python UDF from PIG LATIN. I am doing an ASCII to Binary conversion and wrote a script in python which works in the python shell but if we call it in PIG as a Python UDF, getting an error saying "NameError: global name…
mercuryman
  • 11
  • 3
0
votes
1 answer

Re-evaluate non volatile UDF automatically

Say, we have the following non-volatile UDF: Function Twice(ByRef x As Double) As Double Application.Volatile False Twice = 2 * x End Function which we enter in a cell A1 with reference to cell A2, i.e. in A1 we put: = Twice(A2) …
Confounded
  • 446
  • 6
  • 19
0
votes
1 answer

Calling UDF in ASP.NET

I am a newbie in ASP.NET, having trouble in how to call an inline User Defined Function in my ASP.NET web application. Here, I have passed two arguments in my function - one is available leave(lv) and another one is duration (dr). I am simply…
user3325349
  • 51
  • 1
  • 6
0
votes
1 answer

cannot resolve method in Java UDF from Pig

I am using Pig on Hadoop and DataFu sample here (http://datafu.incubator.apache.org/docs/datafu/guide/set-operations.html), here is my code and error message, anyone have any thoughts what is wrong? Thanks. register datafu-1.2.0.jar; define…
Lin Ma
  • 9,739
  • 32
  • 105
  • 175
0
votes
2 answers

Why MySQL UDF Returns Random Data?

Information about the system in which the problem occurs: OS: Debian 8.1 64 bits - MySQL Version: 5.5.44 - GCC: 4.9.2 I'm creating a simple UDF function that will return the string Hello World. The problem is that it will return Hello World, concat…
Jesus Gonzalez
  • 411
  • 6
  • 17