2

Is it Possible to have Multiple Kettle property files with multiple values but same parameter name For example.I am having client_id and client_name defined in kettle properties as 1 and Microsoft and also i wanted to store same parameters but the values are difference client_id=2 client_name=Google in a separate kettle.properties file in a separate folder.So is it possible to have same ETL in multiple folder structure with multiple kettle.properties.So that during execution of each of the two ETL's it will read the parameters from the kettle properties and going to load into the output with different values accordingly.

Coding_line
  • 127
  • 4
  • 15
  • 1
    Yes it is quite possible. Try to read the each property file whenever you are trying to run the etl. It will not work if you are setting it in parallel and executing it. – Rishu Shrivastava Mar 17 '15 at 05:54
  • Absolutely possible. Moreover, this is how you design batch processing dynamically (see answer below). You would not write the same code twice for two different clients. Rather, multiple different configuration files for the very same process. – Yuval Herziger May 15 '15 at 06:55

1 Answers1

1

Absolutely, this is how it works. To put it into shell code, here's an example for the same extraction process with completely different property files in different paths, writing two different log files (example for Linux, the idea for Windows is similar):

Client #1's command:

~/path/to/your/pdi/environment/base/kitchen.sh - \
file=/path/to/your/extraction/Your_main_process_job.kjb - \
param:Extraction.Properties.Filename.Directory=/path/to/client_1_config/ \
> ~/path/to/client_1_log/batch_run_$(date +\%Y\%m\%d_\%H\%M).log&
  • invoking shell script kitchen.sh with job file file Your_main_process_job.kjb
  • with configuration directory client_1_config
  • with output log to client_1_log log folder

Client #2's command:

~/path/to/your/pdi/environment/base/kitchen.sh - \
file=/path/to/your/extraction/Your_main_process_job.kjb - \
param:Extraction.Properties.Filename.Directory=/path/to/client_2_config/ \
> ~/path/to/client_2_log/batch_run_$(date +\%Y\%m\%d_\%H\%M).log&
  • same as above
  • with configuration directory client_2_config
  • with output log to client_2_log log folder

I hope this helps.

Yuval Herziger
  • 1,145
  • 2
  • 16
  • 28