Rank within Groups in Pig 11

Question

Pig question,

I have my data setup the following way.

Function    Group   Home    Name
Rent    MX  1   John
Rent    MX  1   Jake
Rent    MX  1   Pat
Rent    DG  2   Jason
Rent    DG  6   Patrick
Rent    DG  6   Smith
Rent    DG  6   Joe

What I want to do is Group by function,group and home and then rank within that group.

Function    Group   Home    Name    Rank
Rent    MX  1   John    1
Rent    MX  1   Jake    2
Rent    MX  1   Pat 3
Rent    DG  6   Patrick 1
Rent    DG  6   Smith   2
Rent    DG  6   Joe 3

The RANK function in Pig does not allow me to RANK within group.Any suggestions? Jython UDF ?

DMulligan · Answer 1 · 2014-06-16T00:10:12.443

1

Check out the Enumerate UDF in DataFu, it does this for you. http://datafu.incubator.apache.org/docs/datafu/1.1.0/datafu/pig/bags/Enumerate.html

edited Jun 16 '14 at 00:10

answered Apr 12 '13 at 06:36

DMulligan

8,993
6
33
34

score 0 · Answer 2 · answered Apr 09 '13 at 06:50

0

I will give some pointers to this.

In Cascading API ,I used buffer which allows us to iterate the group values.

I read that cascading also has an api for Jython developers ,you may explore that.

answered Apr 09 '13 at 06:50

Balaswamy Vaddeman

8,360
3
30
40

score 0 · Accepted Answer · answered Apr 09 '13 at 12:06

0

Ok this worked

def num_bag(input):
output = []
for rank, item in num(input):
    output.append(tuple([rank] + list(item)))
return output

answered Apr 09 '13 at 12:06

JohnMeek

69
1
2
6

Rank within Groups in Pig 11

3 Answers3