Questions tagged [data-integration]

Data integration is the combination of technical and business processes used to combine data from disparate sources into meaningful and valuable information. A complete data integration solution encompasses discovery, cleansing, monitoring, transforming and delivery of data from a variety of sources.

Data integration is the combination of technical and business processes used to combine data from disparate sources into meaningful and valuable information. A complete data integration solution encompasses discovery, cleansing, monitoring, transforming and delivery of data from a variety of sources.

It is a huge topic for the IT because, ultimately aims to make all systems work seamlessly together.

Example with data warehouse

The process must take place among the organization's primary transaction systems before data arrives at the data warehouse.
It is rarely complete, unless the organization has a comprehensive and centralized master data management(MDM) system.

Data integration usually takes the form of conforming dimensions and facts in the data warehouse. This means establishing common dimensional attributes across separated databases. Conforming facts is making agreement on common business metrics such as key performance indicators (KPIs) across separated databases, so these numbers can be compared mathematically.

332 questions
0
votes
1 answer

Pentaho Data Integration (Kettle) Supported I/O Formats List

I'm wondering where I can find an official list of supported I/O formats for Pentaho's Data Integration product.
weberc2
  • 7,423
  • 4
  • 41
  • 57
0
votes
1 answer

Update many flag columns given the value of other columns (NULL values) in the same record

Given a table of input data I would like to have a set o "flag columns" that describe sibling columns of the same record. I am using Microsoft SQL Server 2005. First of all I need to load data in the target table, then I need to perform checks to…
TPPZ
  • 4,447
  • 10
  • 61
  • 106
0
votes
1 answer

ETL Pentaho advice (or SQL advice) on convertin CSV into this table format

I have a CSV to convert. Transforming the data into my schema is a bit problematic. I have a locationID in my end table, and in the CSV, there are 1's or a NULL in about 10 columns(each one is a different location). My logic is: If…
williamsandonz
  • 15,864
  • 23
  • 100
  • 186
-1
votes
0 answers

Recommendations for a No-Code ETL Tool with Specific Features

I'm currently searching for a suitable ETL (Extract, Transform, Load) tool that aligns with specific features for my project. I've researched extensively but would greatly appreciate your insights and recommendations on this matter. I'm in need of a…
Arif Alan
  • 1
  • 1
-1
votes
1 answer

How to batch delete records from a airtable base using google apps script?

I have a script that retrieves Album names and NO. of albums in each album using google apps script, and I want to delete all the records in a table before writing new records to that table. Here is the script that I use to retrieve data from google…
-1
votes
1 answer

Transfer xml data to Oracle table by column or fields by using talend

I am using Talend Studio with objects tFileInputDelimited row1(Main) to tOracleOutput what I want is to transfer the data in xml file to Oracle table. I want to transfer the values of the last two columns (product_label and email_order) of my excel…
peace3106
  • 37
  • 4
-1
votes
2 answers

How to multiply rows in data integration studio?

I have one data set looking like this: ID Name YoB ApproximateYob 1 Aa 1980 1980 1 Aa 1980 1979 1 Aa 1980 1981 2 Bb 1990 1991 2 Bb 1990 1989 2 Bb 1990 1990 -And another one like this: ID Name YoB Title Country Alias 1 Aa…
Matthias
  • 1
  • 1
-1
votes
3 answers

Is DataStage Merge stage just a left outer join with multiple other sources?

It appears that the DataStage Merge stage is just a left outer join with the Master being the "left" side and driving input. The other inputs are joined with the master when possible. Is that all there is to it? What am I missing?
lit
  • 14,456
  • 10
  • 65
  • 119
-1
votes
2 answers

Someone have any examples of how to treat return Json (url) using Modified Java Script Value?

Someone have any examples of how to treat return Json (url) using Modified Java Script Value? Follow data the I need from url json: { "result": { "data": [ { "name": "page1", "period": "dia", "values": [ { "value": 4, "end_time":…
-2
votes
1 answer

How to convert '02-3月 -21' to date 21-03-02 00:00:00 psql

How to convert '02-3月 -21' to date 21-03-02 00:00:00? in postgresql or kettle
Elsa
  • 1
  • 1
  • 8
  • 27
-2
votes
2 answers

Why do a periodic full data refresh?

Is there a benefit from doing a periodic full table refresh when you regularly insert/update/delete incrementally? To clarify, this question is in regards to ETL processes.
James
  • 117
  • 2
  • 11
-2
votes
2 answers

Why web service is called a middleware?

Background: wiki says: Services that can be regarded as middleware include enterprise application integration, data integration, message oriented middleware (MOM), object request brokers (ORBs), and the enterprise service bus (ESB). Below is the…
overexchange
  • 15,768
  • 30
  • 152
  • 347
-2
votes
1 answer

Not able to add into mysql database using Talend Job

While adding data into mysql database using Job, I am getting following errors *Starting job fillraw at 16:34 10/03/2016. [statistics] connecting to socket on port 3955 [statistics] connected Data truncation: Out of range value for column…
-2
votes
1 answer

Difference between hadoop and google analytics

Helly guys, I am new to hadoop and everything around big data. while my research about social media data integration with big data i found a lot about hadoop. But I know there is google analytics too, if i want to observe social media and get some…
-3
votes
1 answer

Handling Unstructured Data from Excel in SSIS

I need help with Handling Unstructured Data from Excel. The merged cell data needs to be splitted and set to the respective columns in this case. I have attached the source and target. The Merged Data need to be unmerged and the data need to be…
1 2 3
22
23