0

I have a problem with talend. Currently im using the open source software, and I have a file watcher. I need a way in which even if I enter 2 files at the same second, I want talend to process one at a time, once the first has gone through all the mappings, then the other file can proceed.

I can't use tparrelatize because its not available in the version.

Zuhair Ahmed
  • 17
  • 5
  • tFlowtoIterate component should do the job – Neo Mar 11 '19 at 12:24
  • can you explain further please? I want it to be something like stacks or queues. is this possible ? even if I enter 2 folders to my file watcher directory together ? – Zuhair Ahmed Mar 11 '19 at 17:15
  • the thing is lets say I have 2 folders A and B. Each has 3 same files or at least they have some words in common in the file name. For example A has (apples1, banana1 and orange1). Folder B has (apples2, banana2, and orange2). My file watcher is supposed to look inside the Big directory to which A and B are added, and look for example into a file that has the word apples and then map them. If I add the folders A and B separately to the big directory, they map correctly, but if I add them together, the mapping gets messed up. – Zuhair Ahmed Mar 11 '19 at 17:23
  • I want a component in which although A and B are added together, but something that classifies them in a queue for example, and only read the second folder when the first folder is done with all the executions – Zuhair Ahmed Mar 11 '19 at 17:24
  • As long as you have a single file watcher component which reads both directories and it is followed by an iteration, then yes it will act like a queue. – Neo Mar 12 '19 at 05:05
  • but I have many file watchers .. like lets say directory A and B have the same files.. I have many file watchers reading every single file inside A and B ... – Zuhair Ahmed Mar 15 '19 at 12:55
  • Why do you need to have multiple file watchers? You can change the job to have a single watcher and perform all required functionalities after that. – Neo Mar 15 '19 at 14:55
  • but what if for example the directories im looking into (A and B ) have different files with different names... Im using different file watchers so for example one looks into a file that has the word "apple" in the file path and another looks into a file which has the word "banana" in the file path – Zuhair Ahmed Mar 15 '19 at 15:26
  • Then that means you have separate flows any ways so they are separate queues. – Neo Mar 15 '19 at 15:41
  • but the data keeps on messing up when I enter to db.... Especially the context variables im storing ... Im storing the generated pk of one flow in a context variable so that I can use it in another flow... but it keeps on messing up .. like the fk becomes 2 2 3 instead of 1 2 3 in the child table – Zuhair Ahmed Mar 15 '19 at 15:43

0 Answers0