Questions tagged [csv]

Comma-Separated Values or Character-Separated Values (CSV) is a common "flat file database" (or spreadsheet-style) format for storing tabular data in plain text, with fields separated by a special character (comma, tab, etc). Rows are typically denoted by newline characters. Use for any delimited file formats, including tab delimited (TSV)

CSV is a file format involving a plain text file with information separated by delimiters with the purpose of storing data in a table-structured format. CSV (comma separated values) files traditionally and most commonly use a comma delimiter (hence the name), but other characters can be used, such as semi-colons, tabs, pipe symbols (|), etc.

The MIME type for CSV files is text/csv.

Information is often stored in CSV format to make it easy to transfer tables of data between applications. Each row of a table is represented as a list of plain text (human-readable) values with a delimiter character between each discrete piece of data. Values may be enclosed in quotes, which is required if they contain the delimiter as a value. The first row of data often contains headers of table's columns, which describe the meaning of the data in each column.

Example

Tabular format

Time Temperature Humidity Description
08:00 70 35 Sunny and Clear
11:45 94 90 Hazy, Hot, and Humid
14:30 18 Freezing
16:00 -200 "Unliveable"

CSV format

Time,Temperature,Humidity,Description
08:00,70,35,Sunny and Clear
11:45,94,90,"Hazy, Hot, and Humid"
14:30,18,,Freezing
16:00,-200,,""Unliveable""

In this example, the first row of CSV data serves as the "header", which describes the corresponding data below it. There is no inherent way to describe within a CSV file whether the first row is a header row or not. Each successive line of the CSV file should neatly fit into the same field as the first line.

Note:

  • Empty fields (fields with no available data, such as the third field in the last line) are place-held with commas so that the fields that follow may be correctly placed.
  • Since the comma is the delimiter for fields, the commas in the Description field of the second line must be quoted (to prevent them from being interpreted as field delimiters). Wrapping the entire field in double quotes (") is the default method for protecting the delimiter character inside a field.
  • Since the double-quote is the delimiter quote character, double-quotes in the data, as in "Unliveable" on the fourth line, must also be protected. Doubling-up the double-quote is the default method for protecting the quote character inside a field.

Questions tagged are expected to relate to programming in some way, for example, parsing/importing CSV files or creating them programmatically.

Related links:

89606 questions
14
votes
2 answers

Quote all fields in CSV output

@out = File.open("#{File.expand_path("CSV")}/#{file_name}.csv", "w") CSV::Writer.generate(@out) do |csv| csv << ["01", "02", "test"] end @out.close When i run above code it stores the values in CSV as 01, 02. test I want them to store as "01",…
vasu
  • 391
  • 3
  • 13
14
votes
1 answer

pd.read_csv add column named "Unnamed: 0

I have a dataframe with 3 columns. I save with pd.to_csv(filename) and then re-open it with pd.read_csv(filename, index_col=False) But I get a dataframe with 4 columns, with the left-most column called Unnamed:0 that is actually just the row…
Cranjis
  • 1,590
  • 8
  • 31
  • 64
14
votes
1 answer

Read Fernet Key Causes ValueError: Fernet key must be 32 url-safe base64-encoded bytes

In this function I am trying to read a Fernet key from a file, or create one if the file doesn't contain a key. from cryptography.fernet import Fernet import csv with open("Keys.txt","rU") as csvfile: reader=csv.reader(csvfile) KeyFound=0 …
Lyra Orwell
  • 1,048
  • 4
  • 17
  • 46
14
votes
5 answers

How to configure CsvHelper to skip MissingFieldFound rows

public interface ICsvProductReaderConfigurationFactory { Configuration Build(); } public class CsvProductReaderConfigurationFactory : ICsvProductReaderConfigurationFactory { private readonly ClassMap classMap; public…
Makrushin Evgenii
  • 953
  • 2
  • 9
  • 20
14
votes
1 answer

pyspark csv at url to dataframe, without writing to disk

How can I read a csv at a url into a dataframe in Pyspark without writing it to disk? I've tried the following with no luck: import urllib.request from io import StringIO url =…
RobinL
  • 11,009
  • 8
  • 48
  • 68
14
votes
2 answers

How to display image stored in pandas dataframe?

import pandas as pd from scipy import misc import numpy as np import matplotlib.pyplot as plt W = {'img':[misc.imread('pic.jpg')]} df = pd.DataFrame(W) # This displays the image plt.imshow(df.img1[0]) plt.show() df.to_csv('mypic.csv') new_df=…
PiccolMan
  • 4,854
  • 12
  • 35
  • 53
14
votes
3 answers

Rails - CSV export: prompt for file download

I want to give my users the ability to export a table to CSV. So in my controller, I've added on top of the file: respond_to :html, :js, :csv I'm also setting the headers if the requested format is csv: if params[:format] == 'csv' …
Pierre
  • 8,368
  • 4
  • 34
  • 50
14
votes
4 answers

Pandas read csv is shifting columns

I'm trying to create a dataframe of a csv file that has 4 empty columns. When I open it on LibreOffice or Excel it correctly identifies the empty columns. However, opening with pd.read_csv() ends up shifting the columns' values by one. How can I…
Marcos Santana
  • 911
  • 5
  • 12
  • 21
14
votes
6 answers

How to deal with Commas in CSV using Javascript

I am having some data which is facing issue while exporting to csv if the data contains "," with in it. Hereis the code: CSVData.push('"' + item.OrgId+ '","' + item.Name + '","' + item.OrgDescriptionString + '","' + itam.OrgDetailsString + '","' +…
AMDI
  • 895
  • 2
  • 17
  • 40
14
votes
0 answers

get column names from csv file using pandas

I am using pandas to read a heavy csv file as below import pandas csv_file_path = "/Users/import.csv" csv_data = pandas.read_csv(csv_file_path, names=data_mapping_field_values) When i use csv_data.columns, i got below Index([u'Domain',…
Shiva Krishna Bavandla
  • 25,548
  • 75
  • 193
  • 313
14
votes
3 answers

Embed csv in html rmarkdown

Example Problem statement Solution search and Question ... see rmarkdown example code. Appreciate answers demonstrating the solution by modifying the rmarkdown snippet. --- title: "Reproducable Example" author: "user2030503" output:…
user2030503
  • 3,064
  • 2
  • 36
  • 53
14
votes
1 answer

Remove file after Flask serves it

I have a Flask view that generates data and saves it as a CSV file with Pandas, then displays the data. A second view serves the generated file. I want to remove the file after it is downloaded. My current code raises a permission error, maybe…
cnstlungu
  • 547
  • 1
  • 9
  • 22
14
votes
8 answers

How to check if a CSV has a header using Python?

I have a CSV file and I want to check if the first row has only strings in it (ie a header). I'm trying to avoid using any extras like pandas etc. I'm thinking I'll use an if statement like if row[0] is a string print this is a CSV but I don't…
plshelp
  • 243
  • 1
  • 3
  • 12
14
votes
1 answer

TypeError: argument 1 must have a "write" method

My problem at hand is to get the data from API's and input that data into csv. I'm able to get the data, however outputting the data into csv is where I'm getting the error. Can someone please help. Here is the sample code: import csv,sys def…
user4943236
  • 5,914
  • 11
  • 27
  • 40
14
votes
4 answers

Export R data.frame to SPSS

There is a package foreign with a function write.foreign() that can write a SPS and CSV file. The SPS file than can read the CSV fiel into SPSS including labels. Fine so far, but there are some issues with that function: Newer SPSS versions may…
BurninLeo
  • 4,240
  • 4
  • 39
  • 56