Questions tagged [data-integration]

Data integration is the combination of technical and business processes used to combine data from disparate sources into meaningful and valuable information. A complete data integration solution encompasses discovery, cleansing, monitoring, transforming and delivery of data from a variety of sources.

Data integration is the combination of technical and business processes used to combine data from disparate sources into meaningful and valuable information. A complete data integration solution encompasses discovery, cleansing, monitoring, transforming and delivery of data from a variety of sources.

It is a huge topic for the IT because, ultimately aims to make all systems work seamlessly together.

Example with data warehouse

The process must take place among the organization's primary transaction systems before data arrives at the data warehouse.
It is rarely complete, unless the organization has a comprehensive and centralized master data management(MDM) system.

Data integration usually takes the form of conforming dimensions and facts in the data warehouse. This means establishing common dimensional attributes across separated databases. Conforming facts is making agreement on common business metrics such as key performance indicators (KPIs) across separated databases, so these numbers can be compared mathematically.

332 questions
0
votes
2 answers

Error opening transformations in Pentaho DI in Mac

I have installed Pentaho DI and it works for me but not always. Pentaho DI lets me open only 3 transformations and when I try to open one more (in the menu) the window does not appear. Also when I push a button who opens a dialog box it appears me…
japmelian
  • 97
  • 3
  • 9
0
votes
1 answer

PDI jobs not seen as Mapreduce jobs in Resource Manager or Job History server

I am using Pentaho 5.4 and EMR 3.4 When I execute a transformation in Pentaho to copy data from Oracle DB to HDFS on EMR, I don't see any MR jobs in Resource manager of the Hadoop(EMR) cluster. Am I supposed to see them as MR jobs or pentaho just…
0
votes
1 answer

Pentaho DI Server Unable to Read Repository

I am unable to read the repository when opening the spoon tool.. (Data Integration tool) Unable to read repository with id [PentahoEnterpriseRepositor]. RepositoryMeta is not available. 2015/11/02 13:04:26 - RepositoriesMeta - Reading repositories…
0
votes
2 answers

Data Integragration with Pentaho Kettle

I have three input components: Excel Input XML Input Table Input I would like to do some validation and verification before inserting the integrated data into a data base. Which component should I use for the data transformation/verification?
Rafael Angarita
  • 777
  • 10
  • 27
0
votes
2 answers

Pentaho : Error while running pan.bat file in cmd line

I am trying to run my .ktr file in cmd line. I have my data-integration setup in this path: C:\Users\dhamodharan.a\Desktop\pdi-ce-4.4.0-stable\data-integration and my .ktr file in this path: C:\Users\dhamodharan.a\Desktop\test.krt while am trying…
Dhamodharan
  • 133
  • 1
  • 3
  • 16
0
votes
1 answer

Pentaho Kettle and Scheduling Tool

I am using Pentaho Data Integration 5.2 CE and wish to use a scheduling tool with web UI and power to send alerts as mails in case of success or failures. Please suggest a good open source tool for the purpose.
Devi Prasad
  • 78
  • 2
  • 13
0
votes
1 answer

ODI Sql Spool with GetOption of Query Separator

Hello . I need to run a select using a separator for the fields to make a sql spool WHEN use static character ; So I do: select < % = odiRef.getColList ( " " , " [ EXPRESSION ] ',' || ' , ' || " , " " , " " ) % > from < % = snpRef.getFrom ( ) %…
0
votes
1 answer

A SAS value cannot be converted to a Teradata date

I've got a temporary work table with a date variable source_datetime in SAS DIS. This variable is in the DATETIME22.6 format. I have a teradata table with a date field target_date (type DATE), and using a table loader I am attempting to map…
Rookatu
  • 1,487
  • 3
  • 21
  • 50
0
votes
1 answer

Pentaho cookies with Rest Client Transformation entry

Is there an option to set cookies while using rest client in Pentaho 5.1? I read a couple of blogs and it wasnt mentioned anywhere. I have tried using curl using shell job entry. Got the cookie and used it in my next curl to get data. I need to do…
Rasika Vijay
  • 395
  • 1
  • 7
  • 17
0
votes
1 answer

Set Table Input data to Polling Folder - Pentaho Data Integration

I have a requirement where we can get list of file names from SQL and need to pass these file names as variable to Step which can poll folder for these file names as text file. Please advise how to set SQL output of file names as array variable and…
Madhup Srivastava
  • 446
  • 1
  • 6
  • 18
0
votes
1 answer

Pentaho Text file out separator

I am using a Text file output step in Pentaho Kettle for extracting data from sql and putting into CSV files. I have specified comma as the content separator. But sometimes I receive the files with semicolon seperated values. Any body else has faced…
Karthi
  • 708
  • 1
  • 19
  • 38
0
votes
1 answer

Structured and unstructured data integration with large scale data processing engine

How do data processing engine like Spark, apache flink integrate structured, semi-structured and unstructured data together and affect computation?
user4658980
0
votes
1 answer

updating day by day data in database using pentaho spoon

I have a SQL query as mentioned below and this will update DB every day comparing the date (latest appended date data will be updated in DB) and I want to do this in transformations using Pentaho Data Integration (Kettle). Do While i <…
0
votes
0 answers

Pentaho combining keys of 3 tables

I have 3 tables. The tables are of hierarchial nature. For instance: Table 1 : Contains state information Table2 : City Information Table 3 : Locality Information The Key in Table1 is set as 1,2,3, and there is a foreign key constrain on Table2…
Rasika Vijay
  • 395
  • 1
  • 7
  • 17
0
votes
5 answers

Missing plugins found while loading a transformation on Kettle

I receive this error whenever I run my extraction from the command line, not in the Spoon UI. Missing plugins found while loading a transformation Step : MongoDbInput at…
user962206
  • 15,637
  • 61
  • 177
  • 270