Stack Overflow
Stack Overflow

Questions tagged [data-integration]

Data integration is the combination of technical and business processes used to combine data from disparate sources into meaningful and valuable information. A complete data integration solution encompasses discovery, cleansing, monitoring, transforming and delivery of data from a variety of sources.

Data integration is the combination of technical and business processes used to combine data from disparate sources into meaningful and valuable information. A complete data integration solution encompasses discovery, cleansing, monitoring, transforming and delivery of data from a variety of sources.

It is a huge topic for the IT because, ultimately aims to make all systems work seamlessly together.

Example with data warehouse

The process must take place among the organization's primary transaction systems before data arrives at the data warehouse.
It is rarely complete, unless the organization has a comprehensive and centralized master data management(MDM) system.

Data integration usually takes the form of conforming dimensions and facts in the data warehouse. This means establishing common dimensional attributes across separated databases. Conforming facts is making agreement on common business metrics such as key performance indicators (KPIs) across separated databases, so these numbers can be compared mathematically.

332 questions
0
votes
0 answers

BIRT Eclipse - No records when previewing results

I have the following code in my open script for my dataset: maximoDataSet = MXReportDataSetProvider.create(this.getDataSource().getName(), this.getName()); maximoDataSet.open(); var sqlText = new String(); // Add query to sqlText…
Karen
  • 123
  • 2
  • 4
  • 14
0
votes
0 answers

data integration-, multiple databases, unique incremental SOR_id using talend

I'm trying to integrate multiple databases using talend and in turn have an SOR_id for each table for auditing purposes. is it possible to map between multiple source tables simultaneously to destination table having an SOR_id which is meant to be…
Vishal Ghorpade
  • 1
0
votes
1 answer

SAS pass through - Extract from MySQL does not work

I'm trying to build a Data Integration job uses pass through to extract data from a view in a MySQL database. Wev'e been using pass through a lot in the project, mostly extracting data from Redshift, however with MySQL I was not able to do make it…
user2518751
  • 685
  • 1
  • 10
  • 20
0
votes
1 answer

What are the general procedures to solve data integration challenges in a real Big Data project based on HDFS?

Currently, I'm working on a data mining project, which processes the data stored on the hdfs. The first thing for this project is to integrate data from different databases or tables into uniform tables stored in hdfs. By reading the Data…
Walden Lian
  • 95
  • 2
  • 10
0
votes
0 answers

Setting rows as fieldnames in pentaho

I am importing an excel file that imports as the following: Highlighted in red is what I want to appear as the fieldnames and first row. However as you can see it creates a new row for field names and keeps in the irrelivant information (not…
Karen
  • 123
  • 2
  • 4
  • 14
0
votes
2 answers

informatica-powercenter-better way to implement SQL “between” logic

I have to implement a SQL statment with more than one subquery in Informatica PowerCenter. In one of this I am stuck because I would use as suggested in another answer,2 Source Qualifier and then a Joiner Component, but I need to have the between…
Carmine Tambascia
  • 1,628
  • 2
  • 18
  • 32
0
votes
1 answer

Loops in Pentaho - is this transformation looping?

Is the following transformation looping through each of the rows in the applications field? Also is there a way to loop through and output each individual row to it's own txt or excel file (preferably txt). At the moment it outputs all rows to the…
Karen
  • 123
  • 2
  • 4
  • 14
0
votes
1 answer

Pentaho writing to log but not text file

I have a transformation that is successfully writing the first row to the log file. However the same transformation is not writing the first row to a text file. The text file remains blank. Does anyone know why this may be? edited - only focusing…
Karen
  • 123
  • 2
  • 4
  • 14
0
votes
1 answer

create view over 2 databases into a third one

I have rwo Databases, let's say DB1 and DB2 which DB2 is a copy of DB1 and exactly the same.I also have an empty third Database named Main.How can i create a view within the Main database from the tables of DB1 and DB2.For example if…
soeldiablo
  • 169
  • 6
0
votes
0 answers

Building an engine in Pentaho

Basically I want to create an engine that reads in all data sources and pushes them out. I'll need to use a flag to turn applications on and off and a sequence column to adjust which is running first. It will be a file based repository. Does anyone…
Karen
  • 123
  • 2
  • 4
  • 14
0
votes
1 answer

Talend DI changing Excel cells colour

is there a way in Talend Open Studio for Data Integration (v. 6.3) to set the background colour of cells in a generated Excel file? I looked for some information regarding this but I haven't been able to find a good answer (apart from some people…
LisekKL
  • 131
  • 2
  • 12
0
votes
1 answer

CDA kettle over kettleTransFromFile diffren behaviour vs Pentaho Data Integration

I create in pentaho server a form using CDE. This form is a table with some input fields. On button click is generate an array which is send as parameter value. IN db table i have 3 columns: alfa, beta, gamma. //var data = JSON.stringify(array); var…
DGA
  • 258
  • 3
  • 15
0
votes
2 answers

Merging on different keys into one field in pentaho

I am inserting two excel files into pentaho. Each excel file has multiple sheets. For example I am merging asset data. Each asset has a unique key. When merging using a full outer join, because there are no matches on the keys, a new field is…
Karen
  • 123
  • 2
  • 4
  • 14
0
votes
1 answer

How can I create daily Pentaho reports automatically?

I want to create a report with Pentaho everyday without doing it manually. The report should integrate data from a SQL database. How I´m supposed to do that?
Zyks
  • 1
  • 1
0
votes
0 answers

using eventhub, streaming analytics as a data integration Azure cloud solution

Currently I have a SSIS based data integration solution that extracts data from SQL/Sybase/Oracle/Flat files into staging DB and moves it to our OLTP database with some data massaging. I started researching Azure cloud based solutions to be ready…
Stan Snigir
  • 31
  • 2
1 2 3
…
22 23