0

I want to design a batch job to process multiple zip files in the folder. Basically, the input zip file contains a directory structure and last directory have CSV file and set of PDFs. The job should take zip file and unzip and upload to an external system and database based on the index file in the leaf node folder.

Ex: input zip file structure
input1.zip
--Folder 1
     --> Folder2
         --> abc.pdf
         ...
         ...
         ...
         --> cdf.pdf
         --> metadata.csv
        

I can add spring integration and invoke the job just after the FTP coping completed. However, My question is, how should I design the job to pick up multiple zip files and allow them to process in parallelly.

Since each zip file takes around 10 min to process, I need multiple instances to process zip files in an efficient manner.

Appreciate any suggestions. Thank you.

Kosala Subasinghe
  • 1
  • 1
  • 1
    Is there a reason you want to process all the files by same batch job? Or Could running the multiple instances of the same job is an option? – Prabhakar D Feb 09 '19 at 01:26
  • Program will assign running number for those input pdfs. these number should be in sequence per folder and program store this number in staging table during the processing. if we spawn multiple instance of same job, there is high chance to allocate same number to different pdfs in different folders and sequence will be break. – Kosala Subasinghe Feb 13 '19 at 17:25

0 Answers0