Stack Overflow
Stack Overflow

Questions tagged [flat-file]

A flat-file database model is any system using a single file for storage with one record per line, usually plain-text or text mixed with binary data.

A "flat file" is a plain text or mixed text and binary file which usually contains one record per line or 'physical' record (example on disc or tape). Within such a record, the single fields can be separated by delimiters, e.g. commas, or have a fixed length. In the latter case, padding may be needed to achieve this length. Extra formatting may be needed to avoid delimiter collision. There are no structural relationships between the records.

Typical examples of flat files are /etc/passwd and /etc/group on Unix-like operating systems. Another example of a flat file is a name-and-address list with the fields Name, Address, and Phone Number.

Source: Wikipedia (Flat file database)

830 questions
14
votes
8 answers

Simple, fast SQL queries for flat files

Does anyone know of any tools to provide simple, fast queries of flat files using a SQL-like declarative query language? I'd rather not pay the overhead of loading the file into a DB since the input data is typically thrown out almost immediately…
plinehan
  • 700
  • 7
  • 17
14
votes
2 answers

When is it appropriate to use a database , in Python

I am making a little add-on for a game, and it needs to store information on a player: username ip-address location in game a list of alternate user names that have came from that ip or alternate ip addresses that come from that user name I read…
user1155844
  • 581
  • 1
  • 5
  • 10
12
votes
1 answer

"The selected data source is on remote computer" error when creating an SSIS job

I am in the process of developing an SSIS job on my workstation which a Data Flow Task open a comma delimited flat file on my local workstation and imports the data to a SQL server database on a remote server. When I try to select the destination…
Michael Kniskern
  • 24,792
  • 68
  • 164
  • 231
12
votes
6 answers

How do I get SSIS Data Flow to put '0.00' in a flat file?

I have an SSIS package with a Data Flow that takes an ADO.NET data source (just a small table), executes a select * query, and outputs the query results to a flat file (I've also tried just pulling the whole table and not using a SQL select). The…
theog
  • 1,064
  • 2
  • 20
  • 33
11
votes
4 answers

Flat file vs database - speed?

I'm making a chat program, and I need a place to store messages. The client would contact the server every x seconds with the last received message id, and the server would find all messages with a id higher than that, in the rooms that the client…
apscience
  • 7,033
  • 11
  • 55
  • 89
11
votes
4 answers

where does the term flat-file come from?

I was wondering if anyone knows. Is 'flat' the opposite of hierarchal? Does it come from a phrase like, "this is a flat-out regular file"?
Igor Serebryany
  • 3,307
  • 3
  • 29
  • 41
10
votes
3 answers

SSIS is dropping a record on flat file source import

am experiencing a very strange issue in SSIS (2008). Basic workflow is as follows.. Using a flatfile source (CSV), bring into SSIS, push into SQL. When process is run on dev environment, everything works perfectly. When the dtsx package is placed in…
Grant
  • 11,138
  • 32
  • 94
  • 140
10
votes
2 answers

Is it better for Spark to select from hive or select from file

I was just wondering what people's thoughts were on reading from Hive vs reading from a .csv file or a .txt file or an .ORC file, or a .parquet file. Assuming the underlying Hive table is an external table that has the same file format, would you…
uh_big_mike_boi
  • 3,350
  • 4
  • 33
  • 64
10
votes
19 answers

What arguments to use to explain why SQL Server is far better than a flat file

The higher-ups in my company were told by good friends that flat files are the way to go, and we should switch from SQL Server to them for everything we do. We have over 300 servers and hundreds of different databases. From just the few I'm involved…
jamone
  • 17,253
  • 17
  • 63
  • 98
9
votes
4 answers

Looking for an open-source flatfile/xml database C++ library

I'm looking for a light-weight database library that I can compile into a C++ application. Does any such exist?
Alterlife
  • 6,557
  • 7
  • 36
  • 49
9
votes
1 answer

How to import a flat file without changing regional settings on the deployment server?

I am trying to read a text file in SSIS (2005/2008). I created a connection to the file and placed flat file source, and it works fine in preview but when I try to run I get error saying: [Derived Column [91848]] Error: SSIS Error Code…
Rupal
  • 482
  • 4
  • 8
  • 14
8
votes
6 answers

Is there any way to reorder fields in an SSIS flat file source?

I have an SSIS package using a tab delimited flat file source with a TON of fields. Recently the provider of the tab delimited flat file has decided to change the format of the flat file by sprinkling a couple dozen new fields at random into the…
JKMajcen
  • 962
  • 1
  • 8
  • 7
8
votes
2 answers

Can NHibernate persist to flat files instead of database?

Here is a curiousity question. I have an application that must support both flat files and the database as a place to persist data. I was thinking maybe using a .csv or tab-delimited set of files as input ... Is it possible to use NHibernate to…
Jeffrey Cameron
  • 9,975
  • 10
  • 45
  • 77
8
votes
1 answer

Using FlatFileItemReader with a TaskExecutor (Thread Safety)

There are a lot of examples which use FlatFileItemReader along with TaskExecutor. I provide samples below (both with XML and Java Config): Using Oracle Coherence with Spring Batch Spring Batch Multithreading Example I have used it my self with XML…
kmandalas
  • 418
  • 1
  • 5
  • 17
8
votes
1 answer

Retaining NULLs in numerical columns using SSIS Import/Export Wizard?

I am having a problem uploading data from tab-delimited flat files (TSV files) into SQL Server 2005 using the SSIS Data Import wizard. I did not experience this problem using the equivalent procedure in SQL Server 2000, and I have checked that the…
Lethanta
  • 113
  • 1
  • 6
1
2
3
…
55 56