Questions tagged [csv]

Comma-Separated Values or Character-Separated Values (CSV) is a common "flat file database" (or spreadsheet-style) format for storing tabular data in plain text, with fields separated by a special character (comma, tab, etc). Rows are typically denoted by newline characters. Use for any delimited file formats, including tab delimited (TSV)

CSV is a file format involving a plain text file with information separated by delimiters with the purpose of storing data in a table-structured format. CSV (comma separated values) files traditionally and most commonly use a comma delimiter (hence the name), but other characters can be used, such as semi-colons, tabs, pipe symbols (|), etc.

The MIME type for CSV files is text/csv.

Information is often stored in CSV format to make it easy to transfer tables of data between applications. Each row of a table is represented as a list of plain text (human-readable) values with a delimiter character between each discrete piece of data. Values may be enclosed in quotes, which is required if they contain the delimiter as a value. The first row of data often contains headers of table's columns, which describe the meaning of the data in each column.

Example

Tabular format

Time Temperature Humidity Description
08:00 70 35 Sunny and Clear
11:45 94 90 Hazy, Hot, and Humid
14:30 18 Freezing
16:00 -200 "Unliveable"

CSV format

Time,Temperature,Humidity,Description
08:00,70,35,Sunny and Clear
11:45,94,90,"Hazy, Hot, and Humid"
14:30,18,,Freezing
16:00,-200,,""Unliveable""

In this example, the first row of CSV data serves as the "header", which describes the corresponding data below it. There is no inherent way to describe within a CSV file whether the first row is a header row or not. Each successive line of the CSV file should neatly fit into the same field as the first line.

Note:

  • Empty fields (fields with no available data, such as the third field in the last line) are place-held with commas so that the fields that follow may be correctly placed.
  • Since the comma is the delimiter for fields, the commas in the Description field of the second line must be quoted (to prevent them from being interpreted as field delimiters). Wrapping the entire field in double quotes (") is the default method for protecting the delimiter character inside a field.
  • Since the double-quote is the delimiter quote character, double-quotes in the data, as in "Unliveable" on the fourth line, must also be protected. Doubling-up the double-quote is the default method for protecting the quote character inside a field.

Questions tagged are expected to relate to programming in some way, for example, parsing/importing CSV files or creating them programmatically.

Related links:

89606 questions
12
votes
3 answers

How to properly document text/csv responses on swagger/yaml docs?

I have a bunch of API endpoints that return text/csv content in their responses. How do I document this? Here is what I currently have: /my_endpoint: get: description: Returns CSV content parameters: - $ref:…
theJuls
  • 6,788
  • 14
  • 73
  • 160
12
votes
3 answers

How can I append data to a file on google cloud storage

I am creating a CSV file on google cloud storage using google cloud function. Now I want to edit that file - is it possible to append data in that file? If yes, then how?
12
votes
5 answers

Importing CSV File to Google Maps

I have an quite big CSv File I want to have in Google Maps or just on a map. These are just coordinates but I have 600.000 of them.. Do you have any Idea how I can do this? I've added an screenshot from XTabulator below:
woodleader
  • 923
  • 1
  • 5
  • 14
12
votes
2 answers

Reading a CSV file using Python

please tell me what's the problem in this code it's giving an error import csv with open('some.csv', 'rb') as f: reader = csv.reader(f) for row in reader: print row
atul
  • 183
  • 1
  • 1
  • 7
12
votes
1 answer

Issue importing CSV file using FileHelpers

I'm using FileHelpers library to import csv files into the database table. I'm having an issue importing the file that has the field that starts with the number ('800NUMBER') in the file header. Code to import: DataTable data =…
OlegR
  • 195
  • 1
  • 10
12
votes
1 answer

python csv module error

When I use Pythons csv module, it shows me "delimiter" must be an 1-character string" My code is like this sep = "," srcdata = cStringIO.StringIO(wdata[1]) data = csv.reader(srcdata, delimiter=sep) wdata[1] is a string source. How do I fix…
alwx
  • 179
  • 2
  • 9
12
votes
2 answers

How to create a CSV file using Swift

In an app I created to collect data from Apple pencil input, I tried to export the data into a CSV file. But so far, I only managed to create a single column which records the time length. I want to add another column to record the force from the…
Fëanor Tang
  • 151
  • 1
  • 2
  • 8
12
votes
3 answers

Pyspark read csv with schema, header check, and store corrupt records

I'm trying to use pyspark csv reader with the following criteria: Read csv according to datatypes in schema Check that column names in header and schema matches Store broken records in a new field Here is what I have tried. file:…
Foldager
  • 479
  • 1
  • 4
  • 10
12
votes
6 answers

Importing CSV that has line breaks within the actual fields

I am using PHP to import a CSV file, which originates from an excel spreadsheet. Some of the fields contain line breaks in them, so when I reopen the csv in excel / open office spreadsheet, it misinterprets where the line breaks should happen. Also…
Horse
  • 3,023
  • 5
  • 38
  • 65
12
votes
9 answers

AWS Glue Crawler Cannot Extract CSV Headers

At my wits end here... I have 15 csv files that I am generating from a beeline query like: beeline -u CONN_STR --outputformat=dsv -e "SELECT ... " > data.csv I chose dsv because some string fields include commas and they are not quoted, which…
Mac
  • 1,143
  • 6
  • 21
  • 45
12
votes
6 answers

Reading data from CSV into dataframe with multiple delimiters efficiently

I have an awkward CSV file which has multiple delimiters: the delimiter for the non-numeric part is ',', for the numeric part ';'. I want to construct a dataframe only out of the numeric part as efficiently as possible. I have made 5 attempts: among…
jpp
  • 159,742
  • 34
  • 281
  • 339
12
votes
2 answers

R write.csv with UTF-16 encoding

I'm having trouble outputting a data.frame using write.csv using UTF-16 character encoding. Background: I am trying to write out a CSV file from a data.frame for use in Excel. Excel Mac 2011 seems to dislike UTF-8 (if I specify UTF-8 during text…
Daniel Dickison
  • 21,832
  • 13
  • 69
  • 89
12
votes
2 answers

Athena unable to parse date using OpenCSVSerde

I have a very simple csv file on S3 "i","d","f","s" "1","2018-01-01","1.001","something great!" "2","2018-01-02","2.002","something terrible!" "3","2018-01-03","3.003","I'm an oil man" I'm trying to create a table across this using the following…
Kirk Broadhurst
  • 27,836
  • 16
  • 104
  • 169
12
votes
3 answers

Python 2 and 3 csv reader

I'm trying to use the csv module to read a utf-8 csv file, and I have some trouble to create a generic code for python 2 and 3 due to encoding. Here is the original code in Python 2.7: with open(filename, 'rb') as csvfile: csv_reader =…
Syl
  • 2,733
  • 2
  • 17
  • 20
12
votes
1 answer

Handling quotes with FileHelpers

I'm using the excellent FileHelpers library to import many csv files, but run into a problem. I have a csv file with these three example lines id,text,number 120,"good line this one",789 121,""not good" line", 4456 122,,5446 and this (example)…
edosoft
  • 17,121
  • 25
  • 77
  • 111