Questions tagged [csv]

Comma-Separated Values or Character-Separated Values (CSV) is a common "flat file database" (or spreadsheet-style) format for storing tabular data in plain text, with fields separated by a special character (comma, tab, etc). Rows are typically denoted by newline characters. Use for any delimited file formats, including tab delimited (TSV)

CSV is a file format involving a plain text file with information separated by delimiters with the purpose of storing data in a table-structured format. CSV (comma separated values) files traditionally and most commonly use a comma delimiter (hence the name), but other characters can be used, such as semi-colons, tabs, pipe symbols (|), etc.

The MIME type for CSV files is text/csv.

Information is often stored in CSV format to make it easy to transfer tables of data between applications. Each row of a table is represented as a list of plain text (human-readable) values with a delimiter character between each discrete piece of data. Values may be enclosed in quotes, which is required if they contain the delimiter as a value. The first row of data often contains headers of table's columns, which describe the meaning of the data in each column.

Example

Tabular format

Time Temperature Humidity Description
08:00 70 35 Sunny and Clear
11:45 94 90 Hazy, Hot, and Humid
14:30 18 Freezing
16:00 -200 "Unliveable"

CSV format

Time,Temperature,Humidity,Description
08:00,70,35,Sunny and Clear
11:45,94,90,"Hazy, Hot, and Humid"
14:30,18,,Freezing
16:00,-200,,""Unliveable""

In this example, the first row of CSV data serves as the "header", which describes the corresponding data below it. There is no inherent way to describe within a CSV file whether the first row is a header row or not. Each successive line of the CSV file should neatly fit into the same field as the first line.

Note:

  • Empty fields (fields with no available data, such as the third field in the last line) are place-held with commas so that the fields that follow may be correctly placed.
  • Since the comma is the delimiter for fields, the commas in the Description field of the second line must be quoted (to prevent them from being interpreted as field delimiters). Wrapping the entire field in double quotes (") is the default method for protecting the delimiter character inside a field.
  • Since the double-quote is the delimiter quote character, double-quotes in the data, as in "Unliveable" on the fourth line, must also be protected. Doubling-up the double-quote is the default method for protecting the quote character inside a field.

Questions tagged are expected to relate to programming in some way, for example, parsing/importing CSV files or creating them programmatically.

Related links:

89606 questions
14
votes
3 answers

Building array of objects from parsed csv files in node

I have multiple csv files of the form model1A model1B model2A model2B where each csv is an array i.e. model1A = [1, 1, 1] I want to parse these csvs and create a single array containing all these models, where each element in the array is an…
Philip O'Brien
  • 4,146
  • 10
  • 46
  • 96
14
votes
5 answers

Convert CSV file to XML

I need to Convert a CSV into an XML document. The examples I have seen so far, all show how to do this with a fixed number of columns in the CSV. I have this so far, using LINQ: String[] File = File.ReadAllLines(@"C:\text.csv"); String xml…
Soeren
  • 1,023
  • 4
  • 22
  • 41
14
votes
3 answers

How to import CSV to Trello

I have a .csv file that I want to import to Trello. How can I go about doing that? Zapier won´t work because there are too many and I would be required to pay, which I´m trying to avoid.
tisokoro
  • 265
  • 1
  • 3
  • 13
14
votes
1 answer

fread (data.table in R) with specification of encoding

Could not find proper answer in previous questions and answers to my problem: 1. I have a 2.3 GB csv file which contains 2.4 million rows of Hebrew Text, currently coded in ASCII. Since we are talking about big file, fread would be preferable but…
Dmitry Leykin
  • 485
  • 1
  • 7
  • 14
14
votes
4 answers

Postgres: \copy syntax error in .sql file

I'm trying to write a script that copies data from a crosstab query to a .csv file in Postgres 8.4. I am able to run the command in the psql command line but when I put the command in a file and run it using the -f option, I get a syntax error.…
David Kelley
  • 1,418
  • 3
  • 21
  • 35
14
votes
5 answers

How to programmatically guess whether a CSV file is comma or semicolon delimited

In most cases, CSV files are text files with records delimited by commas. However, sometimes these files will come semicolon delimited. (Excel will use semicolon delimiters when saving CSVs if the regional settings has the decimal separator set as…
Polemarch
  • 1,646
  • 14
  • 10
14
votes
3 answers

'utf-8' codec can't decode byte 0x89

I want to read a csv file and process some columns but I keep getting issues. Stuck with the following error: Traceback (most recent call last): File "C:\Users\Sven\Desktop\Python\read csv.py", line 5, in for row in reader: File…
Sven
  • 371
  • 1
  • 6
  • 17
14
votes
1 answer

converting a d3.csv method to d3.csv.parse method

I just need to draw a d3 barchart of data retrieved from an sql query, so I don't have a tsv or csv file but a string of data in csv format. I know I can use d3.csv.parse method but somehow I couldn't figure out how to convert the example code for…
Baki
  • 141
  • 1
  • 1
  • 4
14
votes
1 answer

Adding BOM to CSV file using fputcsv

I have a simple CSV file being generated that includes foreign characters. I've noted that if I don't include a Byte Order Mark that the foreign characters aren't appearing properly in Excel (but they appear fine when a BOM is present). How can I…
Chuck Le Butt
  • 47,570
  • 62
  • 203
  • 289
14
votes
2 answers

Is it possible to include csv file as part of python package

Is it possible to include a csv file as part of python package? I am creating a package and want some default config files which are imported at runtime. I know I can store as a list or other structure in a .py file, but this will break the pattern…
Ammar Akhtar
  • 1,698
  • 3
  • 13
  • 25
14
votes
4 answers

Comma within fields in CSV file -import to DB using SSIS

I have a CSV file - it has many values with comma as a part of them. The commas within the fields will mislead my SSIS package making it seem like the row has more columns than previously. How to resolve this? Eg: Name,Amount,Address Me,20,000,My…
Murali Uppangala
  • 884
  • 6
  • 22
  • 49
14
votes
7 answers

How to easily process CSV file to List

In my application I use a lot of CSV files which I have to read and build a lists based on them. I'd like to discover an easy way to do this. Do you know any easy framework which does it without using number of config files etc? For instance, I have…
user3279067
14
votes
4 answers

How to add CsvHelper records to DataTable to use for SqlBulkCopy to the database

I am trying to read a CSV file with CsvHelper, load each record into a DataTable, and then use SqlBulkCopy to insert the data into a database table. With the current code, I get an exception when adding a row to the DataTable. The exception is:…
Justin Nafe
  • 3,002
  • 2
  • 18
  • 16
14
votes
3 answers

Read a single column of a CSV and store in an array

What is the best way to read from a csv, but only one specific column, like title? ID | date| title | ------------------- 1| 2013| abc | 2| 2012| cde | The column should then be stored in an array like this: data = ["abc", "cde"] This…
dh762
  • 2,259
  • 4
  • 25
  • 44
14
votes
5 answers

Big csv file c++ parsing performance

I have a big csv file (25 mb) that represents a symmetric graph (about 18kX18k). While parsing it into an array of vectors, i have analyzed the code (with VS2012 ANALYZER) and it shows that the problem with the parsing efficiency (about 19 seconds…
squeezy
  • 607
  • 1
  • 7
  • 15