Pig udf on Filter

Question

I have a use-case in which i need to take in the date of a month to return the previous month's last date.

Ex: input:20150331 output:20150228

I will be using this previous month's last date to filter a daily partition(in pig script).

B = filter A by daily_partition == GetPrevMonth(20150331);

I have created an UDF(GetPrevMonth) which takes the date and returns the previous month's last date.But unable to use it on the filter.

ERROR:Could not infer the matching function for GetPrevMonth as multiple or none of them fit. Please use an explicit cast.

My udf takes tuple as input. Googling it says that UDF cannot be applied on filters. Is there any workaround? or am i going wrong somewhere?

UDF:public class GetPrevMonth extends EvalFunc<Integer> {

    public Integer exec(Tuple input) throws IOException {
        String getdate = (String) input.get(0);
        if (getdate != null){
        try{
            //LOGIC to return prev month date
        }

Need help.Thanks in advance.

You should accept Balduz's answer unless you feel that it isn't satisfactory (to me it seems right) — Eyal, May 19 '16 at 10:39

Balduz · Accepted Answer · 2015-07-28T15:09:38.480

You can call a UDF in a FILTER, but you are passing a number to the function while you expect it to receive a String (chararray inside Pig):

String getdate = (String) input.get(0);

The simple solution would be to cast it to chararray when calling the UDF:

B = filter A by daily_partition == GetPrevMonth((chararray)20150331);

Generally, when you see some error like Could not infer the matching function for X as multiple or none of them fit, 99% of the time the reason is that the values you are trying to pass to the UDF are wrong.

One last thing, even if it is not necessary, in a future you might want to write a pure FILTER UDF. In that case, instead of inheriting from EvalFunc, you need to inherit from FilterFunc and return a Boolean value:

public class IsPrevMonth extends FilterFunc {
    @Override
    public Boolean exec(Tuple input) throws IOException {
        try {
            String getdate = (String) input.get(0);
            if (getdate != null){   
                //LOGIC to retrieve prevMonthDate

                if (getdate.equals(prevMonthDate)) {
                    return true;
                } else {
                    return false;   
                }
            } else {
                return false;
            }
        } catch (ExecException ee) {
            throw ee;
        }
    }
}

Pig udf on Filter

1 Answers1