Set FTP file download limit in pentaho job

Question

I have a pentaho job which contain "get file with ftp" entry which download all file from remote server to local. How can i limit the file download to a specified number, ie:if 300 files are there in remote directory i want to download only 50 random files in each execution.

No direct property, you have to create some logic for this. – Helping Hand.. Aug 06 '18 at 10:12 — Helping Hand.., Aug 06 '18 at 10:12
can you please suggest one logic for this.@WorkingHard.. – Nandu Aug 06 '18 at 10:46 — Nandu, Aug 06 '18 at 10:46

score 0 · Answer 1 · answered Aug 06 '18 at 15:48

0

I did similar specific ftp requirements like yours from a User Java class step in the past.

Its the only way to achieve what you want.

answered Aug 06 '18 at 15:48

jacktrade

3,125
2
36
50

score 0 · Answer 2 · answered Aug 08 '18 at 19:55

Unfortunately the JOB steps for downloading FTP files work only with a RegExp, which will download all files one by one that match, OR, by passing a filename/URI one by one.

If you need a pause between the execution of a group of N files, then there is way to achieve it, you would use a Transformation Executor to segregate a group of N files, and then use a JOB Executor to pass each file of this group to the JOB step, you would then implement the pause you want before each group of 50 files.

Set FTP file download limit in pentaho job

2 Answers2