0

I am newbie to Talend, currently i have designed a job to read files of a certain pattern using tfilelist and put into hadoop. Is there any way to send the rejected files to another folder. Also I tried with providing regex for not choosing a certain pattern of file, but it seems not working. eg: - Like I have 4 files,

 1) bd20.101.log
 2) bd20.102.log
 3) bd20.103.log
 4) bd200.100.log

I want to get only the regex which gives me 4th type which is not a similar pattern.

Viki888
  • 2,686
  • 2
  • 13
  • 16
Deepan Ram
  • 842
  • 1
  • 10
  • 25

1 Answers1

0

Yes you can in a Tmap output option. You need to create a second output. One you put your normal output and a second with the reject.

In you tfilelist you can put a filter to read all your file with a patern like "bd20*" (the star wil take all the file start with bd20).

then in yout tmap you do an output with the parameter reject like the screen.

Then in the path of your tfileoutput you pass the dynamic variable of your tfilelist :

  • "C:\" + ((String)globalMap.get("tFileList_1_CURRENT_FILE"))

enter image description here

Théo Capdet
  • 1,042
  • 3
  • 18
  • 34
  • But for this you have to implement the filter in tMap itself and not in filelist, so you want tFileList to read ALL the files then tMap to split. You may also want to add a tIterateToFlow between the filelist and tmap – Balazs Gunics Jan 18 '17 at 09:54
  • Thanks all. I have resolved my issue. I used file mask in advanced settings to remove the unwanted files. Would definitely try the given options in the near future. – Deepan Ram Jan 26 '17 at 16:38