Stack Overflow
Stack Overflow

Questions tagged [flat-file]

A flat-file database model is any system using a single file for storage with one record per line, usually plain-text or text mixed with binary data.

A "flat file" is a plain text or mixed text and binary file which usually contains one record per line or 'physical' record (example on disc or tape). Within such a record, the single fields can be separated by delimiters, e.g. commas, or have a fixed length. In the latter case, padding may be needed to achieve this length. Extra formatting may be needed to avoid delimiter collision. There are no structural relationships between the records.

Typical examples of flat files are /etc/passwd and /etc/group on Unix-like operating systems. Another example of a flat file is a name-and-address list with the fields Name, Address, and Phone Number.

Source: Wikipedia (Flat file database)

830 questions
2
votes
1 answer

Why are log files commonly flat?

Are log files not meant to be read by machines but by users only? I wonder if there are file appenders for any logging framework that write their output to XML.
mamuesstack
  • 1,111
  • 2
  • 16
  • 34
2
votes
1 answer

SSIS Exporting to Flat File Destination (CSV) - Custom Property EscapeQualifier Not Working (Undocumented?)

Many questions have been asked on this topic, but I can't find anything specifically addressing what I see in Visual Studio 2017 (SSDT). A Custom Property named "EscapeQualifier" exists for a flat-file destination component in an SSIS project.…
RichardB
  • 21
  • 3
2
votes
2 answers

Creating Fixed Width File in R with unique positions for each element

I have data structure(list(Animal = c("dog", "cat", "dog ", "cat"), Fur = c("no", "no", "yes", "yes"), Color = c("Pearl", "Midnight ", "Midnight ", "Pearl")), class = "data.frame", row.names = c(NA, -4L)) I am trying to create a fixed with text…
rj44
  • 165
  • 7
2
votes
2 answers

Amazon MWS Flat File Modify Listing

I'm trying to find the documentation for how the flat file looks for modifying the quantity of a product on Amazon. This is what we send at the moment but it would be good to see what the list of headings we can use. SKU | Quantity 000 | 1 I'm…
Vinz
  • 29
  • 5
2
votes
1 answer

Fast search on first field from a large flat text file

I am working on a language database, and for my work, I often need to load 2 entries out of a 10+ GB text file, which contains over 500k entries. This used to be manageable for work when my text file was 1 GB, but now it takes nearly 4 minutes to do…
Michael Douma
  • 1,144
  • 8
  • 21
2
votes
1 answer

SSIS error The column delimiter for column "x" was not found

I have a CSV file that looks like below In my SSIS package (Visual Studio 2017), I create a Flat File connection Text qualifer: " Header Row Delimiter: {LF} Header Rows to skip: 0 Column Delimiter for all columns: Comma {,}, except for the last…
faujong
  • 949
  • 4
  • 24
  • 40
2
votes
2 answers

Convert Delimiter fille to JSON format using Dataweave in Mule

I need to convert existing delimiter file to json format using dataweave in mule. Sample Input: Name~#~ID~#~Company~#~Address SRI~#~1~#~Infy~#~Bangalore Rahul~#~2~#IBM~#~US John~#~3~#~SF~#~UK Sample outPut { Name: Sri ID: 1 Company: Infy Adress:…
Sri Rockz
  • 85
  • 1
  • 9
2
votes
1 answer

How to Update MySQL Database with CSV(flat-file) and validate data before insertion

I have about 40K records that I need to update into my php/mysql application on a daily basis. What is the recommended best approach? I did some research on load data in file to load the flat-file in a temp MySQL table and run validation on the…
user204245
  • 243
  • 1
  • 3
  • 14
2
votes
1 answer

How to find which flat file contains data errors while loading multiple flat files using Foreach File enumerator in SSIS

I have a question in SSIS. For an instance, I have 100 flat files of the same metadata columns to be loaded using an incremental load, but my question is how can we find which flat contains error data while loading using for each loop container. Any…
Sreenu131
  • 2,476
  • 1
  • 7
  • 18
2
votes
1 answer

Converting flat file 8859-1 encoding to OLE DB Destination using codepage 1252

Input flat file is encoded with ISO 8859-1. The database has default codepage 1252. Is the only way to change codepages to use a Data Conversion transformation? Yes, I know that 8859-1 and 1252 are very close, but I am trying to do this formally…
lit
  • 14,456
  • 10
  • 65
  • 119
2
votes
13 answers

Help required with ancient, unknown storage system

Morning all, I've gone and told a customer I could migrate some of their old data out of a DOS based system into the new system I've developed for them. However I said that without actually looking at the files that stored the data in the old system…
djdd87
  • 67,346
  • 27
  • 156
  • 195
2
votes
1 answer

Load big text file in SQL Server

I want to load a very big file 3GB of text ( not comma separated ) just a text so each line in the text become a record my table structure should be like this LoadedTable ID bigint identity TLine varchar(max) I tried using SQL tasks to import data…
asmgx
  • 7,328
  • 15
  • 82
  • 143
2
votes
1 answer

Set datatypes for SSIS Connection Manager object other than manually / one-by-one?

Have a large number of (TSV) files that need to be imported (periodically) (via SSIS package) into existing MSSQL DB tables. Getting many data type issues from the OLE DB Destination tasks eg: [Flat File Source [2]] Error: Data conversion failed.…
lampShadesDrifter
  • 3,925
  • 8
  • 40
  • 102
2
votes
3 answers

how to skip dynamic header rows in ssis

Hi I have one doubt in SSIS, I want load multiple csv files into SQL server table using SSIS package. while loading time we need consider data from headers on wards. Source path have 3 csv files with fixed header columns with data but each file…
gbalu
  • 377
  • 4
  • 11
2
votes
1 answer

SSIS - Is there a way to filter data from a flat table?

I have a general ledger transactions down to the individual journal bookings. I am working on a process to rebuild these booking from the bottoms up using tables and other information that is located in other places. To start, I have an SSIS…
Vaslo
  • 480
  • 1
  • 5
  • 22
1 2 3
…
55 56