1

I have 6 different input datasets. I want to run ETL over all 6 datasets so they all get transformed to the same output table (same columns and types).

I am using Pentaho (Spoon) to do this.

Is there a way I can define an output table schema to be used by all these transformations in Pentaho? I am using MySQL as my output database.

Thanks in advance.

jacktrade
  • 3,125
  • 2
  • 36
  • 50
rustybeanstalk
  • 2,722
  • 9
  • 37
  • 57

1 Answers1

1

Sounds like you need the Select Values step. Put one of those on the last hop of each dataset's path and make the metadata for the paths all look EXACTLY the same. Then you can connect the output from each Select Values step into a Table Output. All the rows from each set will be mixed together in no particular order.

This can be more challenging than it looks. Spoon will throw errors if any of the fields aren't just exactly identical to the corresponding field from all other datasets. You'll have to find some way to get all the metadata from the datasets to be the same.

Brian.D.Myers
  • 2,448
  • 2
  • 20
  • 17