Questions tagged [csv]

Comma-Separated Values or Character-Separated Values (CSV) is a common "flat file database" (or spreadsheet-style) format for storing tabular data in plain text, with fields separated by a special character (comma, tab, etc). Rows are typically denoted by newline characters. Use for any delimited file formats, including tab delimited (TSV)

CSV is a file format involving a plain text file with information separated by delimiters with the purpose of storing data in a table-structured format. CSV (comma separated values) files traditionally and most commonly use a comma delimiter (hence the name), but other characters can be used, such as semi-colons, tabs, pipe symbols (|), etc.

The MIME type for CSV files is text/csv.

Information is often stored in CSV format to make it easy to transfer tables of data between applications. Each row of a table is represented as a list of plain text (human-readable) values with a delimiter character between each discrete piece of data. Values may be enclosed in quotes, which is required if they contain the delimiter as a value. The first row of data often contains headers of table's columns, which describe the meaning of the data in each column.

Example

Tabular format

Time Temperature Humidity Description
08:00 70 35 Sunny and Clear
11:45 94 90 Hazy, Hot, and Humid
14:30 18 Freezing
16:00 -200 "Unliveable"

CSV format

Time,Temperature,Humidity,Description
08:00,70,35,Sunny and Clear
11:45,94,90,"Hazy, Hot, and Humid"
14:30,18,,Freezing
16:00,-200,,""Unliveable""

In this example, the first row of CSV data serves as the "header", which describes the corresponding data below it. There is no inherent way to describe within a CSV file whether the first row is a header row or not. Each successive line of the CSV file should neatly fit into the same field as the first line.

Note:

  • Empty fields (fields with no available data, such as the third field in the last line) are place-held with commas so that the fields that follow may be correctly placed.
  • Since the comma is the delimiter for fields, the commas in the Description field of the second line must be quoted (to prevent them from being interpreted as field delimiters). Wrapping the entire field in double quotes (") is the default method for protecting the delimiter character inside a field.
  • Since the double-quote is the delimiter quote character, double-quotes in the data, as in "Unliveable" on the fourth line, must also be protected. Doubling-up the double-quote is the default method for protecting the quote character inside a field.

Questions tagged are expected to relate to programming in some way, for example, parsing/importing CSV files or creating them programmatically.

Related links:

89606 questions
12
votes
3 answers

Convert from CSV to array in Python

I have a CSV file containing the…
12
votes
2 answers

python pandas read_csv quotechar does not work

I've read this, this and this posts but despite I don't know why quotechar does not work at pd.read_csv() (Python 3, pandas 0.18.0 and 0.18.1). And how could I read a dataframe like this: "column1","column2", "column3", "column4", "column5",…
ragesz
  • 9,009
  • 20
  • 71
  • 88
12
votes
3 answers

Pandas: Creating new data frame from only certain columns

I have a csv file with measurements, and I want to create a new csv file with the hourly averages and standard deviations. But only for certain columns. Example: csv1: YY-MO-DD HH-MI-SS_SSS | Acceleration | Lumx | …
ValientProcess
  • 1,699
  • 5
  • 27
  • 43
12
votes
2 answers

Print OLS regression summary to text file

I am running OLS regression using pandas.stats.api.ols using a groupby with the following code: from pandas.stats.api import ols df=pd.read_csv(r'F:\file.csv') result=df.groupby(['FID']).apply(lambda d: ols(y=d.loc[:, 'MEAN'], x=d.loc[:,…
Stefano Potter
  • 3,467
  • 10
  • 45
  • 82
12
votes
2 answers

How to parse CSV data that contains newlines in field using JavaScript

Given a CSV file that has newline/return characters in certain fields, how do we parse the data without splitting a field into multiple rows. Example CSV Data: ID;Name;Country;ISO-2;Address;Latitude;Longitude 022wje3;Europa;Italy;IT;"Viale Kennedy…
nelsonic
  • 31,111
  • 21
  • 89
  • 120
12
votes
4 answers

Convert XML file to csv file format in c#

I am using accord.net mouse gesture recognition sample application, which saves the file in above xml format. I need help to convert above xml in to CSV format so i can do machine learning using accord.net Dynamic time warping. I can not figure out…
Sohaib Omar
  • 133
  • 1
  • 1
  • 8
12
votes
1 answer

Rows are lost when reading this tab-separated file with pandas read_csv

I have a .text file with following format, where fields (index number, name and message) are separated by \t (tab-separated): 712 ben Battle of the Books 713 james i used to be in TOM 714 tomy i was in BOB once 715 ben Tournaments of…
user4462740
12
votes
2 answers

Python CSV write to file unreadable in Excel (Chinese characters)

I am trying to performing text analysis on Chinese texts. The program is provided below. I got the result with unreadable characters such as 浜烘皯鏃ユ姤绀捐. And if I change the output file result.csv to result.txt, the characters are correct as 人民日报社论. So…
flyingmouse
  • 1,014
  • 3
  • 13
  • 29
12
votes
1 answer

Read csv file hosted on Google Drive

I am trying to access (read into R) a .csv file hosted on Google Drive (NOT a Drive spreadsheet) -- having set file permission to 'publicly shareable'. So based on the shareable URL: sURL <-"https://drive.google.com/file....view?pli=1" I have been…
remi
  • 781
  • 2
  • 13
  • 22
12
votes
3 answers

Python ASCII codec can't encode character error during write to CSV

I'm not entirely sure what I need to do about this error. I assumed that it had to do with needing to add .encode('utf-8'). But I'm not entirely sure if that's what I need to do, nor where I should apply this. The error is: line 40, in…
f00d
  • 581
  • 1
  • 6
  • 21
12
votes
7 answers

In MySQL Workbench, using "Table Data Import Wizard" to import CSV creates empty table

I am attempting to import a csv file into a MySQL table using the Table Data Import Wizard. The sample section at the bottom of the Configure Import Settings screen looks fine and when I run the import, it says all of my entries were loaded…
Erik Hanson
  • 249
  • 1
  • 4
  • 12
12
votes
3 answers

Read csv file in R with double quotes

Suppose I have a csv file looks like this: Type,ID,NAME,CONTENT,RESPONSE,GRADE,SOURCE A,3,"","I have comma, ha!",I have open double quotes",A,"" desired output should be: df <- data.frame(Type='A',ID=3, NAME=NA, CONTENT='I have comma, ha!', …
Bamqf
  • 3,382
  • 8
  • 33
  • 47
12
votes
3 answers

Amazon Redshift - COPY from CSV - single Double Quote in row - Invalid quote formatting for CSV Error

I'm loading a CSV file from S3 into Redshift. This CSV file is analytics data which contains the PageUrl (which may contain user search info inside a query string for example). It chokes on rows where there is a single, double-quote character, for…
Duncan
  • 10,218
  • 14
  • 64
  • 96
12
votes
4 answers

Java CSVReader ignore commas in double quotes

I have a CSV file that I am having trouble parsing. I am using the opencsv library. Here is what my data looks like and what I am trying to achieve. RPT_PE,CLASS,RPT_MKT,PROV_CTRCT,CENTER_NM,GK_TY,MBR_NM,MBR_PID …
KalebD
  • 121
  • 1
  • 1
  • 6
12
votes
5 answers

Convert .CSV to .XLSX using command line

I'm looking for a way to batch-convert a series of .csv files to .xlsx using the command line. I have tried a bunch of different VBScripts that I found but they all seem to be converting .xlsx to .csv and not the other way around. Here is the…
crackruckles
  • 336
  • 2
  • 5
  • 21
1 2 3
99
100