Stack Overflow
Stack Overflow

Questions tagged [flat-file]

A flat-file database model is any system using a single file for storage with one record per line, usually plain-text or text mixed with binary data.

A "flat file" is a plain text or mixed text and binary file which usually contains one record per line or 'physical' record (example on disc or tape). Within such a record, the single fields can be separated by delimiters, e.g. commas, or have a fixed length. In the latter case, padding may be needed to achieve this length. Extra formatting may be needed to avoid delimiter collision. There are no structural relationships between the records.

Typical examples of flat files are /etc/passwd and /etc/group on Unix-like operating systems. Another example of a flat file is a name-and-address list with the fields Name, Address, and Phone Number.

Source: Wikipedia (Flat file database)

830 questions
6
votes
8 answers

C#: Convert COMP-3 Packed Decimal to Human-Readable Value

I have a series of ASCII flat files coming in from a mainframe to be processed by a C# application. A new feed has been introduced with a Packed Decimal (COMP-3) field, which needs to be converted to a numerical value. The files are being…
tsilb
  • 7,977
  • 13
  • 71
  • 98
5
votes
4 answers

How to effectively test a fixed length flat file parser using MSpec?

I have this method signature: List Parse(string[] lines) ITMData has 35 properties. How would you effectively test such a parser? Questions: Should I load the whole file (May I use System.IO)? Should I put a line from the file into a…
Rookian
  • 19,841
  • 28
  • 110
  • 180
5
votes
1 answer

How to load mixed record type fixed width file with two headers into two separate files

I got a task to load a strangely formatted text file. The file contains unwanted data too. It contains two headers back to back and data for each header is specified on alternate lines. Header rows start after ------. I need to read both the header…
user6423884
  • 51
  • 4
5
votes
2 answers

How to import data from .txt file to populate a table in SQL Server

Every day a PPE.txt file with clients data, separated by semicolon and always with the same layout is stored to a specific file directory. Every day someone has to update a specific table from our database based in this PPE.txt. I want to automate…
Artur Carreira
  • 67
  • 1
  • 1
  • 6
5
votes
3 answers

SSIS: Flat File default length

I have to import about 50 different types of files every day. Some of them with a few columns, some inculde up to 250 columns. The Flat File connection always defaults all columns to 50 chars. Some columns can be way longer than 50 chars, and will…
Esteban P.
  • 2,789
  • 2
  • 27
  • 43
5
votes
3 answers

Writing email messages to flat files in Outlook with VBA

I have written a VBA app that opens a folder in outlook and then iterates through the messages. I need to write the message bodies (with some tweaking) to a single flat file. My code is as follows... Private Sub btnGo_Click() Dim objOutlook As…
Craig
  • 11,614
  • 13
  • 44
  • 62
5
votes
6 answers

Sampling Large Data Files

I currently work in the position of Data Warehouse programmer and as such have to put numerous flat files through ETL process. Of course prior to loading the file I have to be aware of its content, the problem is that majority of the files are > 1…
a_person
  • 177
  • 1
  • 2
  • 12
5
votes
1 answer

flat file database system especially for javascript

Is there any flat file database system that works well with javascript such as one in JSON format or similar. I have heard about mongodb, couchdb and others but it seems whole setup of it must be installed on the computer. Of course, I can't use…
Dev01
  • 4,082
  • 5
  • 29
  • 45
5
votes
3 answers

SQL Server Import Wizard treats NULL as literal string 'NULL'

When I attempt to import a .csv comma-delimited flat file into a Microsoft SQL server 2008R2 64-bit instance, for string columns a NULL in the original data becomes a literal string "NULL" and in a numeric column I receive an import error. Can…
Erik Johnson
  • 858
  • 1
  • 7
  • 29
5
votes
3 answers

Clojure: Store and Compile Large Derived Data Structure

I have a large data structure, a tree, that takes up about 2gb in ram. It includes clojure sets in the leaves, and refs as the branches. The tree is built by reading and parsing a large flat file and inserting the rows into the tree. However this…
David Williams
  • 8,388
  • 23
  • 83
  • 171
5
votes
4 answers

SQL Database VS. Multiple Flat Files (Thousands of small CSV's)

We are designing an update to a current system (C++\CLI and C#). The system will gather small (~1Mb) amounts of data from ~10K devices (in the near future). Currently, they are used to save device data in a CSV (a table) and store all these in a…
Oren
  • 2,462
  • 3
  • 18
  • 16
5
votes
2 answers

How LOCK works for writing logs into flat file?

Read concurrency of flat files is almost unlimited (correct me if I'm wrong); but how is the concurrency for write? Consider a simple access log writing (for visits) in PHP as to append a line of access detail ended with \n fopen(); // in append…
Googlebot
  • 15,159
  • 44
  • 133
  • 229
4
votes
2 answers

Fast read-only embedded "database"?

I'm looking to distribute some information to different machines for efficient and extremely fast access without any network overhead. The data exists in a relational schema, and it is a requirement to "join" on relations between entities, but it is…
Scott
  • 4,070
  • 3
  • 21
  • 16
4
votes
1 answer

sql text field vs flat file vs nosql document store

I plan on having a SQL fact table involving a text field which I don't expect to index on (I will only read out the data and very rarely update it). I think this table could get quite large, primarily due to this text field. The rest of the data…
user1080972
  • 41
  • 1
  • 2
4
votes
3 answers

python/bash SQL for tsv flatfiles (No sqlite)

Background: sqlite is great for doing SQL operations on data loaded into databases, but many times in my line of work I need to do selects, joins, and where statements on files that aren't loaded into a database and not necessarily worth the time to…
sequenceGeek
  • 767
  • 1
  • 8
  • 20
1 2 3
…
55 56