Questions tagged [udf]

A user-defined function (UDF) is a function provided by the user of a program or environment, in a context where the usual assumption is that functions are built into the program or environment. Although the term is widely known in Hadoop components such Hive and Pig, it is also used in other contexts such programming languages and some DBMSs.

From the docs:

Introduction

Pig provides extensive support for user defined functions (UDFs) as a way to specify custom processing. Pig UDFs can currently be implemented in three languages: Java, Python, and JavaScript.

The most extensive support is provided for Java functions. You can customize all parts of the processing including data load/store, column transformation, and aggregation. Java functions are also more efficient because they are implemented in the same language as Pig and because additional interfaces are supported such as the Algebraic Interface and the Accumulator Interface.

Limited support is provided for Python and JavaScript functions. These functions are new, still evolving, additions to the system. Currently only the basic interface is supported; load/store functions are not supported. Furthermore, JavaScript is provided as an experimental feature because it did not go through the same amount of testing as Java or Python. At runtime note that Pig will automatically detect the usage of a scripting UDF in the Pig script and will automatically ship the corresponding scripting jar, either Jython or Rhino, to the backend.

537 questions
0
votes
1 answer

mysql UDF return INT instead of STRING

I am having a problem using a User Defined Function in mysql 5.7.13 (Linux) : This function is defined in mysql using : CREATE FUNCTION PHPParse RETURNS STRING SONAME 'php_parser.so' The C (& C++) file are the following : my_bool…
0
votes
1 answer

user defined function doesn't work vba

i have this udf and basically what I want to get is the latest date from a vector (column) that match with other data in other column, here's the code: Option Explicit Public Function GetLastDate(Carrier As String, CarrierVector As Range, DateVector…
Jey
  • 23
  • 1
  • 1
  • 6
0
votes
1 answer

No typeTag exception when trying to pass Map as parameter to Udf

I have been trying to write an UDF. I am receiving an error when I am trying to build the package with SBT. val getUiPropx = udf((prop_map:Map[String, String],prop:String,sub_prop:String,alt_prop:String) => { if (prop_map.contains(prop)) { val…
preitam ojha
  • 239
  • 1
  • 2
  • 7
0
votes
0 answers

Applying scala uaparser to a column of dataframe

How to apply scala uaparser to a column of a dataframe. Each row in the column in the dataframe is of the form - Mozilla/5.0 (iPhone; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9B206…
preitam ojha
  • 239
  • 1
  • 2
  • 7
0
votes
1 answer

Parsing the Parameters of a Function

I am trying to create a UDF within VBA which go through some function syntax and treat it as Text. The function will look like : FunctionA( Param1 , Param2 , Param3 , Param 4 ) I am trying to develop a UDF which will pull out the value of the…
The_BMan
  • 105
  • 11
0
votes
1 answer

Error while calling udf from within withColumn in Spark using Scala

I receive and error while calling udf from within withColumn in Spark using Scala. This error happens while building with SBT. val hiveRDD = sqlContext.sql("select * from iac_trinity.ctg_us_clickstream") hiveRDD.persist() val trnEventDf = hiveRDD …
preitam ojha
  • 239
  • 1
  • 2
  • 7
0
votes
2 answers

Excel User Defined Function evaluates to zero if another workbook is opened

I have created a user defined function (UDF) in Excel VBA, which picks up contents from 2 columns and evaluates to a result. The UDF evaluates correctly when no other workbooks are open, but changes to zero when any other workbooks are opened…
umm
  • 35
  • 1
  • 4
0
votes
1 answer

How to check existed udf's and triggers in cassandra?

I created two udf's and one trigger in Cassandra cqlsh.How to check existed udf's and triggers in cassandra?
Raghavan
  • 313
  • 2
  • 10
0
votes
1 answer

How to make UDF to handle default arguments, together with other required arguments?

I have a python function which takes few required arguments together few default arguments, but when I tried as fzumstein mentioned it doesn't works as expected. What am I doing wrong? def Doublesum(a, b=1): return (a + b)**2 In…
Anbu
  • 108
  • 2
  • 9
0
votes
1 answer

Custom function, that works with structured references in Excel tables

How to write UDF in Visual Basic for Applications that can work with structured references like this: =UDF([Colomn1]) or this: =UDF(if(len([Colomn1])>1,[Colomn1])) Any ideas? @Rory, this function work with range, but not work with reference to…
0
votes
1 answer

How can we write a custom ObjectInspector in Hive for a custom Writable type?

I am trying to write a custom writable type for a complex object type - to control the serialization and de-serialization. And I want to create a ObjectInspector for that, i tried to find out an example for the same. But no suitable example is…
Ajeet
  • 675
  • 1
  • 6
  • 20
0
votes
0 answers

undefined symbol: ib_util_malloc when trying to call a UDF

I created a very simple UDF, compiles fine but when I try to use it, it gives the following error: root@chii:/data/database/bot# /opt/firebird/bin/isql chii.fdb Database: chii.fdb, User: SYSDBA SQL> select YKUDF_PERLWRAPPER('/bin/ls' , '-al') from…
Sophia
  • 192
  • 12
0
votes
1 answer

define input parameters for udf in spark scala

I am working in scala and spark I am defining a udf , here it is def udfcrpentcd = udf((CORP_ENT_CD:String)=>{ state_name match{ case "IL1" if state_name.contains("IL1")=> "IL1" case "OK1" if state_name.contains("OK1")=> "OK1" …
j pavan kumar
  • 369
  • 1
  • 6
  • 14
0
votes
0 answers

How to call a php file using mysql triggers, if hosted on AWS EC2?

I have hosted my application and database on AWS EC2 server. Now I want to run a php script using my sql triggers, I tried using UDFs but it seems that UDFs require permissions of accessing the O/S but AWS doesn't allow.
0
votes
1 answer

Hive UDF's treatment of URLs

I've created a Hive UDF that parses a URL. The URL contains query parameters. When I parse the input in my UDF, however, characters like '=' and '&' are converted to gibberish. Initially, I was relying on String's toString() method to convert the…
okello
  • 601
  • 10
  • 27