How to use string as input for csv reader without storing it to file

Question

I'm trying to loop through rows in a csv file. I get csv file as string from a web location. I know how to create csv.reader using with when data is stored in a file. What I don't know is, how to get rows using csv.reader without storing string to a file. I'm using Python 2.7.12.

I've tried to create StringIO object like this:

from StringIO import StringIO

csv_data = "some_string\nfor_example"
with StringIO(csv_data) as input_file:
    csv_reader = reader(csv_data, delimiter=",", quotechar='"')

However, I'm getting this error:

Traceback (most recent call last):
  File "scraper.py", line 228, in <module>
    with StringIO(csv_data) as input_file:
AttributeError: StringIO instance has no attribute '__exit__'

I understand that StringIO class doesn't have __exit__ method which is called when when finishes doing whatever it does with this object.

My answer is how to do this correctly? I suppose I can alter StringIO class by subclassing it and adding __exit__ method, but I suspect that there is easier solution.

Update:

Also, I've tried different combinations that came to my mind:

with open(StringIO(csv_data)) as input_file:

with csv_data as input_file:

but, of course, none of those worked.

Simply use `input_file = StringIO(csv_data)` instead of the `with` block. How do you download the CSV data from the web? You usually get a file object when downloading, which can be used directly without first storing the whole content in a string variable and then wrapping that variable in `StringIO`. — Sven Marnach, Feb 01 '17 at 10:37
Cannot reproduce in Python 3.5. `StringIO` can be used in a `with`. What is *exactly* your version and the code? (`from StringIO import StringIO` is *bad*...) — Serge Ballesta, Feb 01 '17 at 10:41
@SvenMarnach Yes, I can do it this way, but I'd like to use `with`. I'm using `requests.get` to get data, and then create `csv_data` from `response.text`. — Fejs, Feb 01 '17 at 10:42
@SvenMarnach: StringIO supports the context manager interface and can be used in `with` statements... — Serge Ballesta, Feb 01 '17 at 10:43
@SergeBallesta Sorry, forgot to mention that. I use Python 2.7.12. I've updated the question. And I use *exactly* `from StringIO import StringIO`. — Fejs, Feb 01 '17 at 10:44
@Fejs _Why_ do you want to use `with`? It doesn't do anything useful in this case. — Sven Marnach, Feb 01 '17 at 10:53
@Fejs: Ok I had never used StringIO.StringIO. I use io.BytesIO for 8 bit strings or io.StringIO for unicode ones. Both support the context manager interface, that's why I was surprised... — Serge Ballesta, Feb 01 '17 at 10:54
@SvenMarnach Well, it looked logical to use `with` if I tried to represent `csv_data` as a file. Now, it doesn't seem such a good idea... — Fejs, Feb 01 '17 at 10:57

score 17 · Accepted Answer · answered Feb 01 '17 at 10:51

17

>>> import csv
>>> csv_data = "some,string\nfor,example"
>>> result = csv.reader(csv_data.splitlines())
>>> list(result)
[['some', 'string'], ['for', 'example']]

answered Feb 01 '17 at 10:51

Thomas Lehoux

1,158
9
13

5

This won't work if the string has linefeeds inside of quoted values – swooby Oct 16 '17 at 23:54

score 2 · Answer 2 · answered Feb 01 '17 at 11:00

2

You should use the io module instead of the StringIO one, because io.BytesIO for byte string or io.StringIO for Unicode ones both support the context manager interface and can be used in with statements:

from io import BytesIO
from csv import reader
csv_data = "some_string\nfor_example"
with BytesIO(csv_data) as input_file:
    csv_reader = reader(input_file, delimiter=",", quotechar='"')
    for row in csv_reader:
        print row

answered Feb 01 '17 at 11:00

Serge Ballesta

143,923
11
122
252

I'm getting following error when I try this: `Traceback (most recent call last): File "scraper.py", line 227, in with BytesIO(race_csv_data) as input_file: TypeError: 'unicode' does not have the buffer interface` – Fejs Feb 01 '17 at 11:02
1

@Fejs That means that your `race_csv_data` is a unicode string and you should use `io.StringIO` instead of `io.BytesIO` (but your example used a byte string ;-) ...) – Serge Ballesta Feb 01 '17 at 11:08

score 1 · Answer 3 · answered Feb 01 '17 at 10:54

If you like context managers, you can use tempfile instead:

import tempfile


with tempfile.NamedTemporaryFile(mode='w') as t:
     t.write('csv_data')
     t.seek(0)
     csv_reader = reader(open(t.name), delimiter=",", quotechar='"')

As an advantage to pass string splitlines directly to csv reader you can write file of any size and then safely read it in csv reader without memory issues.

This file will be closed and deleted automatically

How to use string as input for csv reader without storing it to file

3 Answers3

Linked