How can I compute the combination of two fields using Pig script?

Question

I have input like:

(1, (a, b, c))
(2, (e, f, g))

The output I expected is like:

(1, a)
(1, b)
(1, c)
(2, e)
(2, f)
(2, g)

score 0 · Answer 1 · answered Dec 22 '14 at 12:17

0

May be it will help you:

A = LOAD 'data' AS (int:a, t1:tuple(t1a:chararray, t1b:chararray,t1c:chararray));

B = FOREACH A GENERATE a,t1.$0,t1.$1,t1.$2;

C = group B by a;


X = COGROUP C BY a, C BY $0;

DUMP X;

answered Dec 22 '14 at 12:17

Ravi H

596
3
23

score 0 · Answer 2 · answered Dec 22 '14 at 12:25

0

Can you try this?

A = LOAD 'input.txt' USING PigStorage() AS (f1:int,T:tuple(f2:chararray,f3:chararray,f4:chararray));
B = FOREACH A GENERATE f1,FLATTEN(TOBAG(T.f2,T.f3,T.f4));
DUMP B;

answered Dec 22 '14 at 12:25

Sivasakthi Jayaraman

4,724
3
17
27

Seems it doesn't work, the output is still: `(f1, T.f2, T.f3, ...)`. – PSJay Peng Dec 23 '14 at 03:47
I found the solution [here](http://stackoverflow.com/questions/11213567/pivot-table-with-apache-pig). Just need one more `FLATTEN`, thanks. – PSJay Peng Dec 23 '14 at 04:06

score 0 · Answer 3 · answered Dec 19 '16 at 09:37

Step 1: Load input file

1 a,b,c

2 e,f,g

as

crude_input = load '' USING PigStorage() AS (id:int, ip_tuple:tuple(val1:chararray, val2:chararray, val3:chararray));

dump crude_input;

(1,(a,b,c))

(2,(e,f,g))

Step 2:

crude_flatened = foreach crude_input GENERATE id, FLATTEN($1);

This will generate

(1,a,b,c)

(2,e,f,g)

Step 3:

output_data = foreach crude_flatened generate id, FLATTEN(TOBAG(ip_tuple::val1,ip_tuple::val2,ip_tuple::val3));

(1,a)

(1,b)

(1,c)

(2,e)

(2,f)

(2,g)

How can I compute the combination of two fields using Pig script?

3 Answers3