TypeError: a bytes-like object is required, not 'str' in python and CSV

Question

TypeError: a bytes-like object is required, not 'str'

I'm getting the above error while executing the below python code to save the HTML table data in a CSV file. How do I get rid of that error?

import csv
import requests
from bs4 import BeautifulSoup

url='http://www.mapsofindia.com/districts-india/'
response=requests.get(url)
html=response.content

soup=BeautifulSoup(html,'html.parser')
table=soup.find('table', attrs={'class':'tableizer-table'})
list_of_rows=[]
for row in table.findAll('tr')[1:]:
    list_of_cells=[]
    for cell in row.findAll('td'):
        list_of_cells.append(cell.text)
    list_of_rows.append(list_of_cells)
outfile=open('./immates.csv','wb')
writer=csv.writer(outfile)
writer.writerow(["SNo", "States", "Dist", "Population"])
writer.writerows(list_of_rows)

Possible duplicate of [python 3.5: TypeError: a bytes-like object is required, not 'str'](http://stackoverflow.com/questions/33054527/python-3-5-typeerror-a-bytes-like-object-is-required-not-str) — tripleee, Sep 16 '16 at 09:57

dstudeba · Accepted Answer · 2015-12-15T15:31:11.023

433

You are using Python 2 methodology instead of Python 3.

Change:

outfile=open('./immates.csv','wb')

To:

outfile=open('./immates.csv','w')

and you will get a file with the following output:

SNo,States,Dist,Population
1,Andhra Pradesh,13,49378776
2,Arunachal Pradesh,16,1382611
3,Assam,27,31169272
4,Bihar,38,103804637
5,Chhattisgarh,19,25540196
6,Goa,2,1457723
7,Gujarat,26,60383628
.....

In Python 3 csv takes the input in text mode, whereas in Python 2 it took it in binary mode.

Edited to Add

Here is the code I ran:

url='http://www.mapsofindia.com/districts-india/'
html = urllib.request.urlopen(url).read()
soup = BeautifulSoup(html)
table=soup.find('table', attrs={'class':'tableizer-table'})
list_of_rows=[]
for row in table.findAll('tr')[1:]:
    list_of_cells=[]
    for cell in row.findAll('td'):
        list_of_cells.append(cell.text)
    list_of_rows.append(list_of_cells)
outfile = open('./immates.csv','w')
writer=csv.writer(outfile)
writer.writerow(['SNo', 'States', 'Dist', 'Population'])
writer.writerows(list_of_rows)

edited Dec 15 '15 at 15:31

answered Dec 15 '15 at 08:12

dstudeba

8,878
3
32
41

26

For use with the `csv` module, the Python 3 `open` should also have `newline=''` as a parameter [[ref](https://docs.python.org/3.3/library/csv.html?highlight=csv#csv.reader)] – Mark Tolonen Dec 15 '15 at 08:50
2

Change the 'wb' string to 'w' works for me. Thanks a lot – Loc Huynh Nov 20 '17 at 23:19
If you are using a buffer see [vinyll's answer](https://stackoverflow.com/a/41291817/10687003)! – handras Apr 01 '19 at 19:13
hi there - i tried out the code - and got back this:` Traceback (most recent call last): File "/home/martin/.atom/python/examples/bs_gumtree_pl.py", line 20, in UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0' in position 0: ordinal not in range(128) [Finished in 1.415s]` i have no glue what goes on here – zero Mar 25 '20 at 01:00
1

Is there a python2 and python3 compatible way to do this? (I think that whenever a question addresses an issue that results from differences between python2 and python3 it is nice to give a robust version that works in both cases.) – Kvothe Feb 22 '21 at 17:24

score 38 · Answer 2 · answered Dec 22 '16 at 21:20

38

I had the same issue with Python3. My code was writing into io.BytesIO().

Replacing with io.StringIO() solved.

answered Dec 22 '16 at 21:20

vinyll

11,017
2
48
37

happens to me with stringio also – thebeancounter Feb 25 '19 at 15:23
One consideration: `io.StringIO()` is memory greed and can be a headache with large files. – Flavio Jul 23 '19 at 10:52

score 12 · Answer 3 · answered Sep 18 '19 at 06:02

12

just change wb to w

outfile=open('./immates.csv','wb')

to

outfile=open('./immates.csv','w')

answered Sep 18 '19 at 06:02

Sarath Ak

7,903
2
47
48

score 8 · Answer 4 · answered Apr 30 '20 at 18:22

You are opening the csv file in binary mode, it should be 'w'

import csv

# open csv file in write mode with utf-8 encoding
with open('output.csv','w',encoding='utf-8',newline='')as w:
    fieldnames = ["SNo", "States", "Dist", "Population"]
    writer = csv.DictWriter(w, fieldnames=fieldnames)
    # write list of dicts
    writer.writerows(list_of_dicts) #writerow(dict) if write one row at time

score 2 · Answer 5 · answered Jul 29 '17 at 14:36

2

file = open('parsed_data.txt', 'w')
for link in soup.findAll('a', attrs={'href': re.compile("^http")}): print (link)
soup_link = str(link)
print (soup_link)
file.write(soup_link)
file.flush()
file.close()

In my case, I used BeautifulSoup to write a .txt with Python 3.x. It had the same issue. Just as @tsduteba said, change the 'wb' in the first line to 'w'.

answered Jul 29 '17 at 14:36

Yang Li

21
2

When giving an answer it is preferable to give [some explanation as to WHY your answer](http://stackoverflow.com/help/how-to-answer) is the one. In this case how does this answer differ from the accepted answer? – Stephen Rauch Jul 29 '17 at 14:57
@StephenRauch Thank you for your comments. I'm new here and just have started learning Python several weeks ago. I will try to give better answer in the future. – Yang Li Jul 29 '17 at 15:04
You can edit this post, and add more detail. Hit the edit button below and to the left of the post. – Stephen Rauch Jul 29 '17 at 15:07
@StephenRauch Thanks for your tips! – Yang Li Jul 29 '17 at 15:13

TypeError: a bytes-like object is required, not 'str' in python and CSV

5 Answers5

Linked

Related