0

I am working on a project to extract Master Data from a Database that gathers data from many departments, and this should be done using Talend. I did some researches to understand what is Master Data exactly and I have some modest experience in working with Talend, the problem is how to detect unchangeable data (Master Data) ? I can check dates, but what dates exactly? taking example of a sales department, I work based on order date or delivery date and see if the columns keep their values for a long time, but that doesn't sounds right because:

  1. First of all, the values can be updated but not dates.
  2. Secondly, this rule will work only if the database deals with transactions, but if a column has a distinct values this solution won't work, even though if it is a Master Data.
  3. It is easy to deal with it manually and analyze type of column and then decide if it is a master data or not, but this process should be automated.

I hope my idea is clear and I did clarify the problem I am facing, any peace of advise can really help me a lot, if you have any other method in mind which is better, be my guest!

MFR096
  • 19
  • 2
  • Hi - what do you think master data is? Why do you think it doesn't change? – NickW May 29 '21 at 20:46
  • That's the first impression I had, for example employees data is unlikely to be changed during time, that's why I assumed that time will help to identify it, but the biggest question I got is about input columns, should I choose them myself or even that process should be up to the job? – MFR096 May 29 '21 at 21:54

0 Answers0