201

Sample.csv contains the following:

NAME    Id   No  Dept
Tom     1    12   CS
Hendry  2    35   EC
Bahamas 3    21   IT
Frank   4    61   EE

And the Python file contains the following code:

import csv
ifile  = open('sample.csv', "rb")
read = csv.reader(ifile)
for row in read :
    print (row)

When I run the above code in Python, I get the following exception:

File "csvformat.py", line 4, in for row in read : _csv.Error: iterator should return strings, not bytes (did you open the file in text mode?)

How can I fix it?

Pika Supports Ukraine
  • 3,612
  • 10
  • 26
  • 42

6 Answers6

250

You open the file in text mode.

More specifically: 

ifile  = open('sample.csv', "rt", encoding=<theencodingofthefile>)

Good guesses for encoding is "ascii" and "utf8". You can also leave the encoding off, and it will use the system default encoding, which tends to be UTF8, but may be something else.

Lennart Regebro
  • 167,292
  • 41
  • 224
  • 251
  • 5
    Just want to add to this that if you get encoding errors when you try reading/writing from/to a CSV file, adding a particular encoding can help. I just fixed this bug on mine by adding "encoding = 'utf-8'". – covfefe Oct 16 '15 at 22:06
118

The reason it is throwing that exception is because you have the argument rb, which opens the file in binary mode. Change that to r, which will by default open the file in text mode.

Your code:

import csv
ifile  = open('sample.csv', "rb")
read = csv.reader(ifile)
for row in read :
    print (row)

New code:

import csv
ifile  = open('sample.csv', "r")
read = csv.reader(ifile)
for row in read :
    print (row)
Brad Koch
  • 19,267
  • 19
  • 110
  • 137
MMM
  • 1,181
  • 1
  • 7
  • 2
50

In Python3, csv.reader expects, that passed iterable returns strings, not bytes. Here is one more solution to this problem, that uses codecs module:

import csv
import codecs
ifile  = open('sample.csv', "rb")
read = csv.reader(codecs.iterdecode(ifile, 'utf-8'))
for row in read :
    print (row)
Grigoriy Mikhalkin
  • 5,035
  • 1
  • 18
  • 36
  • 4
    Note that this option isn't the safest. If you can use TextIOWrapper, you should. Issue descriptions: [iterdecode eats empty strings](https://stackoverflow.com/a/43904544/1834570) [iterdecode isn't safe with multi-byte characters](https://stackoverflow.com/a/20602013/1834570) The solution: [TextIOWrapper on a csv stream](https://stackoverflow.com/a/51152810/1834570) – kavdev Jan 20 '19 at 16:38
  • 1
    Thanks! was facing this issue on Python3. – Kenny Aires May 09 '20 at 20:37
  • 2
    @Grigoriy Mikhalkin My adulation goes to for the codecs solution, it just resolve an issue streaming a csv file stored in S3, and csv.reader did not like it without codecs. – dimButTries Dec 16 '21 at 15:01
33

Your problem is you have the b in the open flag. The flag rt (read, text) is the default, so, using the context manager, simply do this:

with open('sample.csv') as ifile:
    read = csv.reader(ifile) 
    for row in read:
        print (row)

The context manager means you don't need generic error handling (without which you may get stuck with the file open, especially in an interpreter), because it will automatically close the file on an error, or on exiting the context.

The above is the same as:

with open('sample.csv', 'r') as ifile:
    ...

or 

with open('sample.csv', 'rt') as ifile:
    ...
Russia Must Remove Putin
  • 374,368
  • 89
  • 403
  • 331
  • The `with` statement a.k.a. the context manager has nothing to do with this question, at all! – RayLuo Mar 09 '19 at 05:50
  • 5
    @RayLuo When I'm demonstrating file handling, I'm also going to demonstrate the best practices around it. I do that fairly consistently. If you're new to Python, and you get stuck in an interactive session with a file you can't do anything with, you would have appreciated my advice... – Russia Must Remove Putin Mar 09 '19 at 13:04
11

I had this error when running an old python script developped with Python 2.6.4

When updating to 3.6.2, I had to remove all 'rb' parameters from open calls in order to fix this csv reading error.

Michael Fayad
  • 1,216
  • 1
  • 17
  • 38
2

This is a GOTCHA in Django. The open that comes with filefield always opens the file in byte mode as far as I can tell. You need to open with Python's open instead.

So with avccont_datafile being an instance of a model with a field called datafile = django.db.models.FileField(), The following code....

with avccount_datafile.datafile.file.open('rt') as fp:
    self.avc_data = csv.DictReader(fp)

gives the error

_csv.Error: iterator should return strings, not bytes (did you open the file in text mode?)

but this fixes it

filename = avccount_datafile.datafile.file.name
with open(filename, 'rt') as fp:
        self.avc_data = csv.DictReader(fp)
MagicLAMP
  • 1,032
  • 11
  • 26