double quotes pandas.read_csv

Question

I have large txt file with multiple words and chars and I'm trying to read this file into a pandas dataframe, with each word or char in a different row.

The problem is that " is one of the chars, and the function reads all the words between two " as a single word (because of the quoting).

How can I address this char as another regular char and not as a quoting char? I tried to play with the parameters of the read_csv function but couldn't manage to fix it.

My code now: data = pd.read_csv(filepath, header=None, delimiter = "\t")

Thanks in advance!

Aditya Gupta · Accepted Answer · 2021-05-30T14:42:32.143

0

you can use the parameter quotechar

data = pd.read_csv("a.txt", delim_whitespace=True, header=None,quotechar="~")
print(data.head())

a.txt

abc def xyz
"abc xyz" def

Output

      0     1    2
0   abc   def  xyz
1  "abc  xyz"  def

there are qoutes left this way.

edited May 30 '21 at 14:42

answered May 30 '21 at 14:15

Aditya Gupta

96
8

Wait, this doesn't seem do anything. I am getting same output with and without the param. Sorry. – Aditya Gupta May 30 '21 at 14:22
@abc123 It seems to happen due to `header=None`, putting it in gives me only two columns. instead of three – Aditya Gupta May 30 '21 at 14:25
Yes :( and maybe I wasn't clear. I don't want to ignore the double quotes, I want them to be like a regular char. – abc123 May 30 '21 at 14:27
@abc123 yeah I changed the quotechar to do it, I changed it to tilda, I doubt it will be in the text file. Check the answer now. – Aditya Gupta May 30 '21 at 14:37
@abc123 can you please upvote this answer? I need one reputation to comment :) – Aditya Gupta Jun 11 '23 at 11:06

score 0 · Answer 2 · answered May 30 '21 at 14:28

Try via numpy's genfromtxt() method:

import numpy as np

data=np.genfromtxt('data.csv',dtype='str',delimeter='\t',skip_header=1)

columns=np.genfromtxt('data.csv',dtype='str',delimiter='\t',skip_footer=len(data))

Finally:

df=pd.Dataframe(data=data,columns=columns)

double quotes pandas.read_csv

2 Answers2