Stack Overflow
Stack Overflow

Questions tagged [flat-file]

A flat-file database model is any system using a single file for storage with one record per line, usually plain-text or text mixed with binary data.

A "flat file" is a plain text or mixed text and binary file which usually contains one record per line or 'physical' record (example on disc or tape). Within such a record, the single fields can be separated by delimiters, e.g. commas, or have a fixed length. In the latter case, padding may be needed to achieve this length. Extra formatting may be needed to avoid delimiter collision. There are no structural relationships between the records.

Typical examples of flat files are /etc/passwd and /etc/group on Unix-like operating systems. Another example of a flat file is a name-and-address list with the fields Name, Address, and Phone Number.

Source: Wikipedia (Flat file database)

830 questions
1
vote
2 answers

How to Identify Problematic Data in SSIS CSV to SQL

I have an Data Flow step in an SSIS package that simply reads data from a CSV with 180 or so columns and inserts it into a MS SQL Server table. It works. However, there's a CSV file with 110,000+ and it fails. In the Output window in Visual Studio…
Amarundo
  • 2,357
  • 15
  • 50
  • 68
1
vote
1 answer

Split rows based on defined values for column from csv file

I have data as below into csv file and I want to pick certain data from it using ssis technology I need only those rows from ID column 23145, 876459, 1022 +--------+------+-----------+ | ID | Name | Address | +--------+------+-----------+ | …
Mike
  • 61
  • 6
1
vote
1 answer

Dynamic file creation with BCP Utility

I'm using BCP Utility to copy records out of table before deleting the records. The function is working just fine, however, I need to copy the records to a new file for the every time I delete, instead of override the same file (as it is now). It…
Mindan
  • 979
  • 6
  • 17
  • 37
1
vote
1 answer

SAS group by counters per variable - primary key creation

I have some data which needs to be split into 12 or so different groups, there is no key and the order the data is in is important. The data has a number of groups and those groups have singular and / or nested groups within that. Each group will be…
teelove
  • 73
  • 5
1
vote
2 answers

Add a column with specific text that is not contained in my flat file before being imported into SQL?

Lets say I have a Flat File with 3 columns (EXAMPLE PIPE SEPARATED) This is Flat File Name | LastName | BirthDay Arthur | Tower | 10-10-1990 Now in SSIS how can I add Columns but I want the column to have a specific text/number (value) on…
Arturo
  • 43
  • 5
1
vote
0 answers

Read a flat file in Pentaho Spoon and then export it's metadata into a CSV

I am wondering if it is possible to extract the metadata of a flat file in a CSV using Pentaho Spoon. What I mean by that is for example get a CSV file input step, choose the file you want to read and then somehow get access to the metadata of that…
flufylele
  • 11
  • 2
1
vote
1 answer

Handling truncation error in derived column in data flow task

I have a data flow task which contains a derived column. The derived column transforms a CSV file column, lets say A which is order number, to a data type char with length 10. This works perfectly fine when the text file column is equal to or less…
SSharma
  • 951
  • 6
  • 15
1
vote
1 answer

NULLs when importing flat file - SQL Server

I am trying to import some data from a .csv file in SSMS using the "Import Flat File" option. However, not all the data is being copied. Both data types are set to nvarchar(50). The lines containing just a single value after the semi-colon is…
nix
  • 4,501
  • 5
  • 22
  • 19
1
vote
1 answer

Unpivot data excel with merged column using SSIS

I'm using SSIS to export from Excel to text and in my case, I need to export where the file contains merged columns anyone can help or have suggest for my case? Input Excel A B C D …
Jackson
  • 11
  • 2
1
vote
1 answer

SSIS File transfer task , Copy Files works but Move file property fails the task

SSIS File System Task can copy but can't move the flat files?. I am trying to move the flat files from source to target folder but FST only working for copy. and while moving it throes and error saying that, file is being used by another process. I…
Abhishek Srivastava
  • 13
  • 4
1
vote
1 answer

How to use SSIS to compare single value in File1 header to total row count in File2 (csv)

I'm tasked with creating a step that validates reported value from an external organization with the actuals delivered from same organization. This organization will drop 2 files on our server. File1 contains a single numerical value File2 contains…
SJJ9166
  • 71
  • 1
  • 6
1
vote
2 answers

How to create separate transformed file instead of overwrite ssis output flat file?

I have a dataflow that is used to do transformation of multiple flat files from given folder using for each loop container. I have a flat file again as output file. The problem is that every time I execute the the job only the last file that got…
Kate
  • 445
  • 3
  • 9
  • 22
1
vote
2 answers

Can we compare columns of multiple inputs file to derive a new column in SSIS

I am trying to create a derived column based on columns provided in different input file but unfortunately I keep getting error when I tried to map my Raw_File_1 with Derived Column.The error looks like this: Cannot create a connector. The…
Kate
  • 445
  • 3
  • 9
  • 22
1
vote
1 answer

Removing CR LF improper split line on .txt pipe-delimited flat with Powershell script

Hope all is well! I came across a bit of a tricky issue with a flat file that is exported from Oracle PBCS with some carriage return issues. End users - when inputting data into PBCS - will often press in a specific data field input screen. When…
smomotiu
  • 39
  • 6
1
vote
2 answers

Flat file data extraction through SSIS with column name and data in same rows

I have to extract data from multiple folders. Each folder can have multiple text files. Text files can have multiple rows. If multiple rows are there there would be possibility of each row can have different number of columns like below: File Name:…
Ritesh
  • 401
  • 4
  • 11
  • 19
1 2 3
…
55 56