0

I am trying to achieve column merge of files in a folder using Talend.(Files are local)

Example:- 4 files are there in a folder. ( there could be 'n' number of files also)

Each file would have one column having 100 values.

So after merge, the output file would have 4 or 'n' number of columns with 100 records in it.

Is it possible to merge this way using Talend components ?

Tried with 2 files in tmap , the output records becomes multiplied ( the record in first file * the record in second file ).

Any help would be appreciated.

Thanks.

Deepan Ram
  • 842
  • 1
  • 10
  • 25

1 Answers1

0

You have to determine how to join data from the different files. If row number N of each file has to be matched with row number N of the other files, then you must set a sequence on each of your file, and join the sequences in order to get your result. Careful, you are totally depending on the order of data in each file.

Then you can have this job :

tFileInputdelimited_1 --> tMap_1 --->{tMap_5
tFileInputdelimited_2 --> tMap_2 --->{tMap_5
tFileInputdelimited_3 --> tMap_3 --->{tMap_5
tFileInputdelimited_4 --> tMap_4 --->{tMap_5

In tMaps from 1 to 4, copy the input to the output, and add a "sequence" column (datatype integer) to your output, populate it with Numeric.sequence("IDENTIFIER1",1,1) . Then you have 2 columns in output : your data and a unique sequence. Be careful to use different identifiers for each source.

Then in tMap_5, just join the different sequences, and get your inputColumn.

Corentin
  • 2,442
  • 1
  • 17
  • 28
  • Thanks for the suggestion, but using this, the order of the columns in the file are not maintained. Did a tpivotToColumnsDelimited to merge all records having same Sequence Number. Now trying a way to shuffle the ordering of the columns to merge that for the original one. – Deepan Ram Feb 23 '17 at 10:56