Questions tagged [csv]

Comma-Separated Values or Character-Separated Values (CSV) is a common "flat file database" (or spreadsheet-style) format for storing tabular data in plain text, with fields separated by a special character (comma, tab, etc). Rows are typically denoted by newline characters. Use for any delimited file formats, including tab delimited (TSV)

CSV is a file format involving a plain text file with information separated by delimiters with the purpose of storing data in a table-structured format. CSV (comma separated values) files traditionally and most commonly use a comma delimiter (hence the name), but other characters can be used, such as semi-colons, tabs, pipe symbols (|), etc.

The MIME type for CSV files is text/csv.

Information is often stored in CSV format to make it easy to transfer tables of data between applications. Each row of a table is represented as a list of plain text (human-readable) values with a delimiter character between each discrete piece of data. Values may be enclosed in quotes, which is required if they contain the delimiter as a value. The first row of data often contains headers of table's columns, which describe the meaning of the data in each column.

Example

Tabular format

Time Temperature Humidity Description
08:00 70 35 Sunny and Clear
11:45 94 90 Hazy, Hot, and Humid
14:30 18 Freezing
16:00 -200 "Unliveable"

CSV format

Time,Temperature,Humidity,Description
08:00,70,35,Sunny and Clear
11:45,94,90,"Hazy, Hot, and Humid"
14:30,18,,Freezing
16:00,-200,,""Unliveable""

In this example, the first row of CSV data serves as the "header", which describes the corresponding data below it. There is no inherent way to describe within a CSV file whether the first row is a header row or not. Each successive line of the CSV file should neatly fit into the same field as the first line.

Note:

  • Empty fields (fields with no available data, such as the third field in the last line) are place-held with commas so that the fields that follow may be correctly placed.
  • Since the comma is the delimiter for fields, the commas in the Description field of the second line must be quoted (to prevent them from being interpreted as field delimiters). Wrapping the entire field in double quotes (") is the default method for protecting the delimiter character inside a field.
  • Since the double-quote is the delimiter quote character, double-quotes in the data, as in "Unliveable" on the fourth line, must also be protected. Doubling-up the double-quote is the default method for protecting the quote character inside a field.

Questions tagged are expected to relate to programming in some way, for example, parsing/importing CSV files or creating them programmatically.

Related links:

89606 questions
12
votes
6 answers

Java - Write hashmap to a csv file

I have a hashmap with a String key and String value. It contains a large number of keys and their respective values. For example: key | value abc | aabbcc def | ddeeff I would like to write this hashmap to a csv file such that my csv file contains…
activelearner
  • 7,055
  • 20
  • 53
  • 94
12
votes
3 answers

postgresql csv copy unquoted newline found in data

I have some csv data in excel, and I'm importing it into postgresql. I'm opening the excel csv file with a notepad editor (have tried notepad, wordpad and notepad++) and am then copying/pasting into a remote desktop connection to a linux machine. …
user1475191
  • 121
  • 1
  • 1
  • 5
12
votes
2 answers

Dynamically converting a list of Excel files to csv files in R

I currently have a folder containing all Excel (.xlsx) files, and using R I would like to automatically convert all of these files to CSV files using the "openxlsx" package (or some variation). I currently have the following code to convert one of…
costebk08
  • 1,299
  • 4
  • 17
  • 42
12
votes
5 answers

How do I convert a .tsv to .csv?

Trying to convert a .tsv to a .csv. This: import csv # read tab-delimited file with open('DataS1_interactome.tsv','rb') as fin: cr = csv.reader(fin, delimiter='\t') filecontents = [line for line in cr] # write comma-delimited file (comma…
hannah
  • 889
  • 4
  • 13
  • 27
12
votes
2 answers

How to delete a CSV file in Python

I'm doing a project which requires me to add, delete data that is in a CSV file, the way I have done it is by creating a new CSV file called outfile.csv, which holds all the information from another CSV file called infile.csv (outfile.csv has some…
user4267078
12
votes
6 answers

convert a fixed width file from text to csv

I have a large data file in text format and I want to convert it to csv by specifying each column length. number of columns = 5 column length [4 2 5 1 1] sample observations: aasdfh9013512 ajshdj 2445df Expected Output aasd,fh,90135,1,2 ajsh,dj,…
Ashish
  • 441
  • 1
  • 5
  • 11
12
votes
5 answers

Splitting a List inside a Pandas DataFrame

I have a csv file that contains a number of columns. Using pandas, I read this csv file into a dataframe and have a datetime index and five or six other columns. One of the columns is a list of timestamps (example below with index) CreateDate …
Eric D. Brown D.Sc.
  • 1,896
  • 7
  • 25
  • 37
12
votes
3 answers

PHP generating csv not sending correct new line feeds

I have a script that generates a csv file using the following code: header('Content-type: text/csv'); header('Content-Disposition: attachment; filename="'.date("Ymdhis").'.csv"'); print $content; The $content variable simply contains lines with…
sjw
  • 2,603
  • 5
  • 22
  • 20
12
votes
3 answers

fgetcsv/fputcsv $escape parameter fundamentally broken

Overview fgetcsv and fputcsv support an $escape argument, however, it's either broken, or I'm not understanding how it's supposed to work. Ignore the fact that you don't see the $escape parameter documented on fputcsv, it is supported in the PHP…
quickshiftin
  • 66,362
  • 10
  • 68
  • 89
12
votes
2 answers

how to use lists as values in pandas dataframe?

I have a dataframe that requires a subset of the columns to have entries with multiple values. below is a dataframe with a "runtimes" column that has the runtimes of a program in various conditions: df = [{"condition": "a", "runtimes": [1,1.5,2]},…
user248237
12
votes
7 answers

C# Importing Large Volume of Data from CSV to Database

What's the most efficient method to load large volumes of data from CSV (3 million + rows) to a database. The data needs to be formatted(e.g. name column needs to be split into first name and last name, etc.) I need to do this in a efficiently as…
guazz
  • 121
  • 1
  • 3
12
votes
2 answers

Reading contents of csv file in node.js

I am trying to implement a module in nodejs(just started working in nodejs) which has requirement below as Upload .csv file. Read content of the csv file. Frameworks currently being used for restful api is "express": "~4.2.0" and multer for file…
Mozak
  • 2,738
  • 4
  • 30
  • 49
12
votes
6 answers

python reading in multi-column tsv file with row numbers

What is the cleanest way of reading in a multi-column tsv file in python with headers, but where the first column has no header and instead contains the row numbers for each row? This is apparently a common format from files coming from R data…
719016
  • 9,922
  • 20
  • 85
  • 158
12
votes
2 answers

SQL won't insert null values with BULK INSERT

I have a CSV file and each line looks similar to this: EASTTEXAS,NULL,BELLVILLE AREA,NULL,BELLVILLE AREA,RGP,NULL,NULL,0,NULL,NULL,NULL,1,1,PM,PM Settings,NULL,NULL I couldn't find any examples on how NULL values were supposed to be handled when…
ernest
  • 1,633
  • 2
  • 30
  • 48
12
votes
4 answers

Generate CSV file from rails

I've been reading similar questions, but many of the answers are outdated or not clear enough for me. I'd like to be able to just do something like (in a controller action): respond_to do |format| format.html format.csv end I know I'd then…
Elliot
  • 13,580
  • 29
  • 82
  • 118
1 2 3
99
100