Questions tagged [csv]

Comma-Separated Values or Character-Separated Values (CSV) is a common "flat file database" (or spreadsheet-style) format for storing tabular data in plain text, with fields separated by a special character (comma, tab, etc). Rows are typically denoted by newline characters. Use for any delimited file formats, including tab delimited (TSV)

CSV is a file format involving a plain text file with information separated by delimiters with the purpose of storing data in a table-structured format. CSV (comma separated values) files traditionally and most commonly use a comma delimiter (hence the name), but other characters can be used, such as semi-colons, tabs, pipe symbols (|), etc.

The MIME type for CSV files is text/csv.

Information is often stored in CSV format to make it easy to transfer tables of data between applications. Each row of a table is represented as a list of plain text (human-readable) values with a delimiter character between each discrete piece of data. Values may be enclosed in quotes, which is required if they contain the delimiter as a value. The first row of data often contains headers of table's columns, which describe the meaning of the data in each column.

Example

Tabular format

Time Temperature Humidity Description
08:00 70 35 Sunny and Clear
11:45 94 90 Hazy, Hot, and Humid
14:30 18 Freezing
16:00 -200 "Unliveable"

CSV format

Time,Temperature,Humidity,Description
08:00,70,35,Sunny and Clear
11:45,94,90,"Hazy, Hot, and Humid"
14:30,18,,Freezing
16:00,-200,,""Unliveable""

In this example, the first row of CSV data serves as the "header", which describes the corresponding data below it. There is no inherent way to describe within a CSV file whether the first row is a header row or not. Each successive line of the CSV file should neatly fit into the same field as the first line.

Note:

  • Empty fields (fields with no available data, such as the third field in the last line) are place-held with commas so that the fields that follow may be correctly placed.
  • Since the comma is the delimiter for fields, the commas in the Description field of the second line must be quoted (to prevent them from being interpreted as field delimiters). Wrapping the entire field in double quotes (") is the default method for protecting the delimiter character inside a field.
  • Since the double-quote is the delimiter quote character, double-quotes in the data, as in "Unliveable" on the fourth line, must also be protected. Doubling-up the double-quote is the default method for protecting the quote character inside a field.

Questions tagged are expected to relate to programming in some way, for example, parsing/importing CSV files or creating them programmatically.

Related links:

89606 questions
101
votes
16 answers

how to import csv data into django models

I have some CSV data and I want to import into django models using the example CSV data: 1;"02-01-101101";"Worm Gear HRF 50";"Ratio 1 : 10";"input shaft, output shaft, direction A, color dark green"; 2;"02-01-101102";"Worm Gear HRF 50";"Ratio 1 :…
little_fish
  • 4,169
  • 5
  • 20
  • 8
101
votes
15 answers

How to import a CSV file into a MySQL table

How can I import a CSV file into a MySQL table? I would like for the first row of data be used as the column names. I read How do I import CSV file into a MySQL table?, but the only answer was to use a GUI and not a shell?
lcm
  • 1,737
  • 6
  • 17
  • 40
100
votes
4 answers

Can you encode CR/LF in into CSV files?

Is it possible/legal to somehow encode CR/LF characters into a CSV file? (as part of a CSV standard?) If so how should I encode CR/LF?
Tore Nestenius
  • 16,431
  • 5
  • 30
  • 40
100
votes
12 answers

Dump a mysql database to a plaintext (CSV) backup from the command line

I'd like to avoid mysqldump since that outputs in a form that is only convenient for mysql to read. CSV seems more universal (one file per table is fine). But if there are advantages to mysqldump, I'm all ears. Also, I'd like something I can run…
dreeves
  • 26,430
  • 45
  • 154
  • 229
100
votes
7 answers

Dump all tables in CSV format using 'mysqldump'

I need to dump all tables in MySQL in CSV format. Is there a command using mysqldump to just output every row for every table in CSV format?
Ken
  • 2,849
  • 8
  • 24
  • 23
99
votes
5 answers

Pandas dataframe read_csv on bad data

I want to read in a very large csv (cannot be opened in excel and edited easily) but somewhere around the 100,000th row, there is a row with one extra column causing the program to crash. This row is errored so I need a way to ignore the fact that…
Fonti
  • 1,169
  • 2
  • 9
  • 14
99
votes
5 answers

How to get the number of columns from a JDBC ResultSet?

I am using CsvJdbc (it is a JDBC-driver for csv-files) to access a csv-file. I don't know how many columns the csv-file contains. How can I get the number of columns? Is there any JDBC-function for this? I can not find any methods for this in…
Jonas
  • 121,568
  • 97
  • 310
  • 388
99
votes
22 answers

How can I merge 200 CSV files in Python?

I here have 200 separate CSV files named from SH (1) to SH (200). I want to merge them into a single CSV file. How can I do it?
Chuck
  • 1,051
  • 1
  • 8
  • 6
98
votes
7 answers

Export from sqlite to csv using shell script

I'm making a shell script to export a sqlite query to a csv file, just like this: #!/bin/bash ./bin/sqlite3 ./sys/xserve_sqlite.db ".headers on" ./bin/sqlite3 ./sys/xserve_sqlite.db ".mode csv" ./bin/sqlite3 ./sys/xserve_sqlite.db ".output…
Rorro
  • 1,227
  • 1
  • 11
  • 17
97
votes
3 answers

How to update selected rows with values from a CSV file in Postgres?

I'm using Postgres and would like to make a big update query that would pick up from a CSV file, lets say I got a table that's got (id, banana, apple). I'd like to run an update that changes the Bananas and not the Apples, each new Banana and their…
user519753
  • 1,527
  • 6
  • 18
  • 22
97
votes
10 answers

Can I automatically create a table in PostgreSQL from a csv file with headers?

I'm running PostgreSQL 9.2.6 on OS X 10.6.8. I would like to import data from a CSV file with column headers into a database. I can do this with the COPY statement, but only if I first manually create a table with a column for each column in the CSV…
ihough
  • 1,111
  • 1
  • 8
  • 8
96
votes
7 answers

How do I read and write CSV files?

How do I read the following CSV file? 1,"A towel,",1.0 42," it says, ",2.0 1337,is about the most ,-1 0,massively useful thing ,123 -2,an interstellar hitchhiker can have.,3 How do I write the following data to a CSV file? data = [ (1, "A…
Martin Thoma
  • 124,992
  • 159
  • 614
  • 958
96
votes
3 answers

IndexError: too many indices for array

I know there is a ton of these threads but all of them are for very simple cases like 3x3 matrices and things of that sort and the solutions do not even begin to apply to my situation. So I'm trying to graph G versus l1 (that's not an eleven, but…
Chris
  • 963
  • 1
  • 6
  • 4
96
votes
8 answers

How to import a csv file into MySQL workbench?

I have a CSV file. It contain 1.4 million rows of data, so I am not able to open that csv file in Excel because its limit is about 1 million rows. Therefore, I want to import this file in MySQL workbench. This csv file contains columns like…
vps
  • 1,337
  • 7
  • 23
  • 41
95
votes
11 answers

Read csv from Google Cloud storage to pandas dataframe

I am trying to read a csv file present on the Google Cloud Storage bucket onto a panda dataframe. import pandas as pd import matplotlib.pyplot as plt import seaborn as sns %matplotlib inline from io import BytesIO from google.cloud import…