Questions tagged [csv]

Comma-Separated Values or Character-Separated Values (CSV) is a common "flat file database" (or spreadsheet-style) format for storing tabular data in plain text, with fields separated by a special character (comma, tab, etc). Rows are typically denoted by newline characters. Use for any delimited file formats, including tab delimited (TSV)

CSV is a file format involving a plain text file with information separated by delimiters with the purpose of storing data in a table-structured format. CSV (comma separated values) files traditionally and most commonly use a comma delimiter (hence the name), but other characters can be used, such as semi-colons, tabs, pipe symbols (|), etc.

The MIME type for CSV files is text/csv.

Information is often stored in CSV format to make it easy to transfer tables of data between applications. Each row of a table is represented as a list of plain text (human-readable) values with a delimiter character between each discrete piece of data. Values may be enclosed in quotes, which is required if they contain the delimiter as a value. The first row of data often contains headers of table's columns, which describe the meaning of the data in each column.

Example

Tabular format

Time Temperature Humidity Description
08:00 70 35 Sunny and Clear
11:45 94 90 Hazy, Hot, and Humid
14:30 18 Freezing
16:00 -200 "Unliveable"

CSV format

Time,Temperature,Humidity,Description
08:00,70,35,Sunny and Clear
11:45,94,90,"Hazy, Hot, and Humid"
14:30,18,,Freezing
16:00,-200,,""Unliveable""

In this example, the first row of CSV data serves as the "header", which describes the corresponding data below it. There is no inherent way to describe within a CSV file whether the first row is a header row or not. Each successive line of the CSV file should neatly fit into the same field as the first line.

Note:

  • Empty fields (fields with no available data, such as the third field in the last line) are place-held with commas so that the fields that follow may be correctly placed.
  • Since the comma is the delimiter for fields, the commas in the Description field of the second line must be quoted (to prevent them from being interpreted as field delimiters). Wrapping the entire field in double quotes (") is the default method for protecting the delimiter character inside a field.
  • Since the double-quote is the delimiter quote character, double-quotes in the data, as in "Unliveable" on the fourth line, must also be protected. Doubling-up the double-quote is the default method for protecting the quote character inside a field.

Questions tagged are expected to relate to programming in some way, for example, parsing/importing CSV files or creating them programmatically.

Related links:

89606 questions
12
votes
2 answers

How to merge two cells in CSV file

How to merge two cell in CSV file like as Excel? I want to merge two cell in CSV file file as: Header Id Name mobileNo Sub-Header id first Name last Name countryCode MobNumber
12
votes
4 answers

How to load very large csv files in nodejs?

I'm trying to load 2 big csv into nodejs, first one has a size of 257 597 ko and second one 104 330 ko. I'm using the filesystem (fs) and csv modules, here's my code : fs.readFile('path/to/my/file.csv', (err, data) => { if (err) console.err(err) …
François MENTEC
  • 1,150
  • 4
  • 12
  • 25
12
votes
3 answers

How can I disable quoting in the Python 2.4 CSV reader?

I am writing a Python utility that needs to parse a large, regularly-updated CSV file I don't control. The utility must run on a server with only Python 2.4 available. The CSV file does not quote field values at all, but the Python 2.4 version of…
Carl Meyer
  • 122,012
  • 20
  • 106
  • 116
12
votes
4 answers

UnicodeError: UTF-16 stream does not start with BOM

I have trouble reading the csv file by python. My csv file has Korean and numbers. Below is my python code. import csv import codecs csvreader = csv.reader(codecs.open('1.csv', 'rU', 'utf-16')) for row in csvreader: print(row) First, there was…
Py11
  • 159
  • 1
  • 2
  • 8
12
votes
1 answer

Converting CSV headers to be case insensitive in Ruby

I have a ruby import to DB process that checks header titles against an array called headers. As it stands right now, those headers must be typed exactly the same as they appear in the array. I would like for them to be accepted regardless if they…
Stinga B
  • 123
  • 1
  • 4
12
votes
1 answer

Spring 3 - export data in csv format

I have some data in the DB and would like to be able to export it to a CSV file and provide a link so the user can download it. Is there any mechanism provided by Spring 3 for this? Do you know how could I do that?
tsunade21
  • 3,442
  • 4
  • 25
  • 22
12
votes
1 answer

Example Application of FasterCSV

I'm very new to Ruby on Rails and web programming as well. Can someone show me some example of FasterCSV implementation.
johan
  • 2,319
  • 9
  • 27
  • 37
12
votes
6 answers

CSV to JSON with PHP?

I need to convert a CSV file to JSON on the server using PHP. I am using this script which works: function csvToJSON($csv) { $rows = explode("\n", $csv); $i = 0; $len = count($rows); $json = "{\n" . ' "data" : ['; foreach…
VerizonW
  • 1,785
  • 5
  • 21
  • 27
12
votes
1 answer

Reading semicolon delimited csv

I have the below block of code which uses OpenCSV to read a CSV file and store the 7th column. The problem I face is that I use ; as delimiter in the CSV file but it takes , as delimiter as well. How can I avoid this? Putting "" in CSV is not…
Sourav Mehra
  • 435
  • 2
  • 7
  • 23
12
votes
3 answers

Read large csv file from S3 into R

I need to load a 3 GB csv file with about 18 million rows and 7 columns from S3 into R or RStudio respectively. My code for reading data from S3 usually works like this: library("aws.s3") obj <-get_object("s3://myBucketName/aFolder/fileName.csv") …
Tom
  • 319
  • 1
  • 3
  • 11
12
votes
5 answers

How to input large data into python pandas using looping or parallel computing?

I have a csv file of 8gb and I am not able to run the code as it shows memory error. file = "./data.csv" df = pd.read_csv(file, sep="/", header=0, dtype=str) I would like to split the files into 8 small files ("sorted by id") using python. And…
user7779326
12
votes
1 answer

Parse CSV file which contains a new line to php array

I have a CSV file with: Test1,One line Test2,"Two lines Hello" Test3,One line As you can see, one of the columns has a value which is separated with a new line. To parse this CSV file into an array, I run: $csvArray = array(); $csvData =…
Henrik Petterson
  • 6,862
  • 20
  • 71
  • 155
12
votes
2 answers

unterminated CSV quoted field in Postgres

I'm trying to insert some data into my table using the copy command : copy otype_cstore from '/tmp/otype_fdw.csv' delimiter ';' quote '"' csv; And I have this answer : ERROR: unterminated CSV quoted field There is the line in my CSV file where I…
Alexandre S.
  • 522
  • 2
  • 5
  • 24
12
votes
3 answers

How can I split a Dataset from a .csv file for Training and Testing?

I'm using Python and I need to split my .csv imported data in two parts, a training and test set, E.G 70% training and 30% test. I keep getting various errors, such as 'list' object is not callable and so on. Is there any easy way of doing…
Midi
  • 143
  • 1
  • 1
  • 7
12
votes
4 answers

Convert JSON to CSV using PowerShell

I have a sample JSON-formatted here which converts fine if I use something like: https://konklone.io/json/ I've tried the following code in PowerShell: (Get-Content -Path $pathToJsonFile | ConvertFrom-Json) | ConvertTo-Csv -NoTypeInformation |…
Anders Ekelund
  • 153
  • 1
  • 1
  • 7