Questions tagged [csv]

Comma-Separated Values or Character-Separated Values (CSV) is a common "flat file database" (or spreadsheet-style) format for storing tabular data in plain text, with fields separated by a special character (comma, tab, etc). Rows are typically denoted by newline characters. Use for any delimited file formats, including tab delimited (TSV)

CSV is a file format involving a plain text file with information separated by delimiters with the purpose of storing data in a table-structured format. CSV (comma separated values) files traditionally and most commonly use a comma delimiter (hence the name), but other characters can be used, such as semi-colons, tabs, pipe symbols (|), etc.

The MIME type for CSV files is text/csv.

Information is often stored in CSV format to make it easy to transfer tables of data between applications. Each row of a table is represented as a list of plain text (human-readable) values with a delimiter character between each discrete piece of data. Values may be enclosed in quotes, which is required if they contain the delimiter as a value. The first row of data often contains headers of table's columns, which describe the meaning of the data in each column.

Example

Tabular format

Time Temperature Humidity Description
08:00 70 35 Sunny and Clear
11:45 94 90 Hazy, Hot, and Humid
14:30 18 Freezing
16:00 -200 "Unliveable"

CSV format

Time,Temperature,Humidity,Description
08:00,70,35,Sunny and Clear
11:45,94,90,"Hazy, Hot, and Humid"
14:30,18,,Freezing
16:00,-200,,""Unliveable""

In this example, the first row of CSV data serves as the "header", which describes the corresponding data below it. There is no inherent way to describe within a CSV file whether the first row is a header row or not. Each successive line of the CSV file should neatly fit into the same field as the first line.

Note:

  • Empty fields (fields with no available data, such as the third field in the last line) are place-held with commas so that the fields that follow may be correctly placed.
  • Since the comma is the delimiter for fields, the commas in the Description field of the second line must be quoted (to prevent them from being interpreted as field delimiters). Wrapping the entire field in double quotes (") is the default method for protecting the delimiter character inside a field.
  • Since the double-quote is the delimiter quote character, double-quotes in the data, as in "Unliveable" on the fourth line, must also be protected. Doubling-up the double-quote is the default method for protecting the quote character inside a field.

Questions tagged are expected to relate to programming in some way, for example, parsing/importing CSV files or creating them programmatically.

Related links:

89606 questions
644
votes
36 answers

Excel to CSV with UTF8 encoding

I have an Excel file that has some Spanish characters (tildes, etc.) that I need to convert to a CSV file to use as an import file. However, when I do Save As CSV it mangles the "special" Spanish characters that aren't ASCII characters. It also…
Jeff Treuting
  • 13,910
  • 8
  • 36
  • 47
617
votes
37 answers

Stop Excel from automatically converting certain text values to dates

Does anyone happen to know if there is a token I can add to my csv for a certain field so Excel doesn't try to convert it to a date? I'm trying to write a .csv file from my application and one of the values happens to look enough like a date that…
user16324
613
votes
31 answers

Is it possible to force Excel recognize UTF-8 CSV files automatically?

I'm developing a part of an application that's responsible for exporting some data into CSV files. The application always uses UTF-8 because of its multilingual nature at all levels. But opening such CSV files (containing e.g. diacritics, cyrillic…
Lyubomyr Shaydariv
  • 20,327
  • 12
  • 64
  • 105
599
votes
5 answers

What MIME type should I use for CSV?

I've seen application/csv used and also text/csv. Is there a difference? Does it matter which as long as the request matches something that's available? Are they interchangeable?
Steve Dunn
  • 21,044
  • 11
  • 62
  • 87
568
votes
18 answers

Is there a way to include commas in CSV columns without breaking the formatting?

I've got a two column CSV with a name and a number. Some people's name use commas, for example Joe Blow, CFA. This comma breaks the CSV format, since it's interpreted as a new column. I've read up and the most common prescription seems to be…
buley
  • 28,032
  • 17
  • 85
  • 106
521
votes
29 answers

Dealing with commas in a CSV file

I am looking for suggestions on how to handle a csv file that is being created, then uploaded by our customers, and that may have a comma in a value, like a company name. Some of the ideas we are looking at are: quoted Identifiers (value "," values…
Bob The Janitor
  • 20,292
  • 10
  • 49
  • 72
494
votes
10 answers

Error "(unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape"

I'm trying to read a CSV file into Python (Spyder), but I keep getting an error. My code: import csv data = open("C:\Users\miche\Documents\school\jaar2\MIK\2.6\vektis_agb_zorgverlener") data = csv.reader(data) print(data) I get the following…
Miesje
  • 4,937
  • 3
  • 10
  • 7
487
votes
14 answers

How to export table as CSV with headings on Postgresql?

I'm trying to export a PostgreSQL table with headings to a CSV file via command line, however I get it to export to CSV file, but without headings. My code looks as follows: COPY products_273 to '/tmp/products_199.csv' delimiters',';
Elitmiar
  • 35,072
  • 73
  • 180
  • 229
476
votes
7 answers

Create Pandas DataFrame from a string

In order to test some functionality I would like to create a DataFrame from a string. Let's say my test data looks like: TESTDATA="""col1;col2;col3 1;4.4;99 2;4.5;200 3;4.7;65 4;3.2;140 """ What is the simplest way to read that data into a Pandas…
Emil L
  • 20,219
  • 3
  • 44
  • 65
447
votes
7 answers

How to add pandas data to an existing csv file?

I want to know if it is possible to use the pandas to_csv() function to add a dataframe to an existing csv file. The csv file has the same structure as the loaded data.
Ayoub Ennassiri
  • 4,606
  • 3
  • 13
  • 9
435
votes
21 answers

Reading CSV file and storing values into an array

I am trying to read a *.csv-file. The *.csv-file consist of two columns separated by semicolon (";"). I am able to read the *.csv-file using StreamReader and able to separate each line by using the Split() function. I want to store each column into…
Rushabh Shah
  • 4,401
  • 3
  • 16
  • 4
397
votes
18 answers

View tabular file such as CSV from command line

Anyone know of a command-line CSV viewer for Linux/OS X? I'm thinking of something like less but that spaces out the columns in a more readable way. (I'd be fine with opening it with OpenOffice Calc or Excel, but that's way too overpowered for…
Benjamin Oakes
  • 12,262
  • 12
  • 65
  • 83
393
votes
11 answers

Response Content type as CSV

I need to send a CSV file in HTTP response. How can I set the output response as CSV format? This is not working: Response.ContentType = "application/CSV";
balaweblog
  • 14,982
  • 28
  • 73
  • 95
393
votes
13 answers

Keep only date part when using pandas.to_datetime

I use pandas.to_datetime to parse the dates in my data. Pandas by default represents the dates with datetime64[ns] even though the dates are all daily only. I wonder whether there is an elegant/clever way to convert the dates to datetime.date or…
user1642513
382
votes
5 answers

Pandas read in table without headers

Using pandas, how do I read in only a subset of the columns (say 4th and 7th columns) of a .csv file with no headers? I cannot seem to be able to do so using usecols.
user308827
  • 21,227
  • 87
  • 254
  • 417