Stack Overflow
Stack Overflow

Questions tagged [udf]

A user-defined function (UDF) is a function provided by the user of a program or environment, in a context where the usual assumption is that functions are built into the program or environment. Although the term is widely known in Hadoop components such Hive and Pig, it is also used in other contexts such programming languages and some DBMSs.

From the docs:

Introduction

Pig provides extensive support for user defined functions (UDFs) as a way to specify custom processing. Pig UDFs can currently be implemented in three languages: Java, Python, and JavaScript.

The most extensive support is provided for Java functions. You can customize all parts of the processing including data load/store, column transformation, and aggregation. Java functions are also more efficient because they are implemented in the same language as Pig and because additional interfaces are supported such as the Algebraic Interface and the Accumulator Interface.

Limited support is provided for Python and JavaScript functions. These functions are new, still evolving, additions to the system. Currently only the basic interface is supported; load/store functions are not supported. Furthermore, JavaScript is provided as an experimental feature because it did not go through the same amount of testing as Java or Python. At runtime note that Pig will automatically detect the usage of a scripting UDF in the Pig script and will automatically ship the corresponding scripting jar, either Jython or Rhino, to the backend.

537 questions
0
votes
1 answer

Python UDF for PIG Giving error

I have a Python UDF which converts the data from Hex to string. When I try to call the UDF on multiple fields, I am getting an error. Here is my Python UDF. Script is hex_to_str.py #!/usr/bin/python @outputSchema("field:chararray") def…
DebD
  • 374
  • 1
  • 4
  • 18
0
votes
1 answer

Hive - Remove substring from string

I need to replace substring from a given string with empty string with the substring appearing in different positions of the string. I want to remove the "fruit":"apple" from these possible combinations of the strings and expected the corresponding…
user2727704
  • 625
  • 1
  • 10
  • 21
0
votes
1 answer

Excel VBA UDF Executes in Immediate Window, Fails on Worksheet

UDF "NAV()" is designed to find the correct report on a network drive based on the first argument (always a date), then loop through all worksheets to find a piece of data with the same row as second argument and same column as third argument…
Micah Neely
  • 3
  • 3
0
votes
1 answer

Mysql stored procedure error 1418

I wanted to create a procedure that can register the user with some very very basic security. I made a procedure, and it worked fine: DELIMITER // CREATE PROCEDURE registriraj (vime varchar(50), vpriimek varchar(50), vemail varchar(100), vgeslo…
Matic Šincek
  • 5
  • 5
0
votes
2 answers

"Flattening" a databag in Pig

Suppose I have a bunch of databags generated from a Pig UDF that holds several tuples of Strings. How can I pull all of them out of the databags and simple make each String its own "row" of data. databags = FOREACH data GENERATE…
kk415kk
  • 1,227
  • 1
  • 14
  • 30
0
votes
1 answer

Hive -e and regex replace udf issue

I am working on Hive 0.13 in MapR distro. I am seeing a weird issue when i run the following query in hive using the hive -e option. Below is my query : select regexp_replace('$60,825.48','\\$|\,',''); The above query works fine when I run from the…
K S Nidhin
  • 2,622
  • 2
  • 22
  • 44
0
votes
1 answer

Returning tables in UDF

I am new to SQL Server and I would like to know how to return the exact columns of tables in UDFs. The situation is like this : I am going to query a table and then return it again as table. Example : create function test var t set t = select *…
JanLeeYu
  • 981
  • 2
  • 9
  • 24
0
votes
3 answers

compare 3 ranges instead of 2

Public Function Compare(r1 As Range, r2 As Range) As Long Dim r As Range, v As Variant, v2 As Variant Dim rr As Range For Each r In r1 v = r.Value If v <> 0 And v <> "" Then For Each rr In r2 v2 = rr.Value …
ArthurV
  • 113
  • 2
  • 8
0
votes
1 answer

Strange #VALUE! error

Please have a look at this error: Here are two functions: Public Function CompareExtra(r1 As Range, r2 As Range) As Long Dim r As Integer, v As Variant, v2 As Variant Dim rr1() As String Dim rr As Range rr1 = Split(r1, ",") For r =…
ArthurV
  • 113
  • 2
  • 8
0
votes
0 answers

HIVE UDF error while creating

While creating an UDF for HIVE, I am getting below error: org.apache.ambari.view.hive.client.HiveErrorStatusException: H110 Unable to submit statement. Error while processing statement: FAILED: Execution Error, return code 1 from…
Ashwin Aravind
  • 181
  • 1
  • 7
0
votes
1 answer

Compare two ranges: classic range vs range of comma-delimited values in a single cell

Suppose I've got a range of values: 6 2 4 7 8 7 1 5 1 9 4 6 8 8 1 7 4 2 And separately in a single cell: "1, 4, 9" I need to compare these values in a single cell with the given range. In case if there is a match -…
ArthurV
  • 113
  • 2
  • 8
0
votes
1 answer

User defined function in ms excel 2010 for fetching environ variable user name does not work

I created a user defined function in ms excel 2010 for fetching current logged in user thru environvariable "USERNAME". However, on some machines running ms excel2010 on win7, this function does not update current user name. I had packaged it in a…
aliarshad111
  • 3
  • 4
0
votes
1 answer

Want to write (or use an existing?) UDF to read a file with a multi-character delimiter

I'd like to read a data file in Pig that uses a multi-character delimiter to separate fields (I've no requirement to write files this way). So my Pig Script will look something like: myData = LOAD 'myFile' USING PigStorage(‘~|~’) as (col1:chararray,…
Stormcloud
  • 2,065
  • 2
  • 21
  • 41
0
votes
1 answer

Write a Hive UDF function that takes variable number of args and outpts a JSON blob

I want to write a Hive UDF which takes variable number of parameters (of different types) and output it as a JSON blob (with column name to column values mapping). Select userId, myudf(col2, col3) from TABLE 2; // the output of udf should be…
Anil Padia
  • 513
  • 1
  • 6
  • 12
0
votes
2 answers

Set Cell Formula in ExcelDNA UDF

I am working on a UDF(User defined function) for Excel user, the task seems easy with the help of ExcelDNA. But While I test it in a cell of Excel with following two formula, both show #VALUE!. Need help to get this solved,…
Wayne
  • 55
  • 1
  • 12
1 2 3
…
35 36