PIG : Columns into rows

Question

I have a file contains this :

id_v^id_f^id_s1,id_s2,id_s3,id_s4
id_v1^id_f1^id_s2,id_s3,id_s4
id_v2^id_f2^id_s2,id_s1,id_s4

this file is a "^" delimited csv. i want to normalise it this way using pig :

id_v^id_f^id_s1
id_v^id_f^id_s2
id_v^id_f^id_s3
id_v^id_f^id_s4
id_v1^id_f2^id_s2
id_v1^id_f2^id_s3
id_v1^id_f2^id_s4
...

Can anyone help me ?

Regards

score 1 · Accepted Answer · answered Jul 07 '15 at 17:24

Ok. Let assume this is my file.Lets say the file name is id_v.txt

id_v^id_f^id_s1,id_s2,id_s3,id_s4
id_v1^id_f1^id_s2,id_s3,id_s4
id_v2^id_f2^id_s2,id_s1,id_s4

Below is the Pig code

A = load '/user/cloudera/inputfiles/id_v.txt' using PigStorage('^') as(v:chararray,f:chararray,data:chararray);
B = foreach A generate v,f,FLATTEN(TOKENIZE(data)) as data;
dump B;

Below is the O/P Produced by above code..

(id_v,id_f,id_s1)
(id_v,id_f,id_s2)
(id_v,id_f,id_s3)
(id_v,id_f,id_s4)
(id_v1,id_f1,id_s2)
(id_v1,id_f1,id_s3)
(id_v1,id_f1,id_s4)
(id_v2,id_f2,id_s2)
(id_v2,id_f2,id_s1)
(id_v2,id_f2,id_s4)

Thank you @Surender. Thanks all i need. – Firas kh Jul 08 '15 at 08:35 — Firas kh, Jul 08 '15 at 08:35

PIG : Columns into rows

1 Answers1