ValueError: unsupported format character 'a' (0x61) at index 79

Question

I am trying to scrape the data from a website using beautiful soup4 and python. Here is my code

from bs4 import BeautifulSoup
import urllib2
i = 0
for i in xrange(0,38):
    page=urllib2.urlopen("http://www.sfap.org/klsfaprep_search?page={}&type=1&strname=&loc=&op=Lancer%20la%20recherche&form_build_id=form-72a297de309517ed5a2c28af7ed15208&form_id=klsfaprep_search_form" %i) 
    soup = BeautifulSoup(page.read())
    for eachuniversity in soup.findAll('div',{'class':'field-item odd'}):
        print ''.join(eachuniversity.findAll(text=True)).encode('utf-8')
    print ',\n'
i= i+ 1

I think the problem is in the URL that I have given and in the increment statement. I am able to scrape page by page. But only when I give the xrange.

You might need to [encode('utf-8') sooner](http://stackoverflow.com/a/2627117/1681480) before processing it... Also, your i values are not used (outside the loop) and are you sure you want page zero? — beroe, Oct 19 '13 at 04:55
Just use @falsetru's answer -- the loop seems right (goes up to 37, not 38). I just thought that pages in URLs usually start at one not zero. — beroe, Oct 19 '13 at 05:02

falsetru · Accepted Answer · 2013-10-19T05:02:46.857

2

Reason of the `ValueError`

You're mixing {} formatting with % formatting.

>>> '{}%20la' % 1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: unsupported format character 'a' (0x61) at index 6
>>> '{}%20la'.format(1)
'1%20la'

I recommend you to use {} formatting, because in URL, there are multiple %s.

page=urllib2.urlopen("http://www.sfap.org/klsfaprep_search?page={}&type=1&strname=&loc=&op=Lancer%20la%20recherche&form_build_id=form-72a297de309517ed5a2c28af7ed15208&form_id=klsfaprep_search_form".format(i))

Complete code

You don't need i = 0 and i = i + 1 because for i in xrange(0,38) take care of it.

import urllib2 # Import standard library module first. (PEP-8)

from bs4 import BeautifulSoup

for i in xrange(0,38):
    page = urllib2.urlopen("http://www.sfap.org/klsfaprep_search?page={}&type=1&strname=&loc=&op=Lancer%20la%20recherche&form_build_id=form-72a297de309517ed5a2c28af7ed15208&form_id=klsfaprep_search_form" .format(i))
    soup = BeautifulSoup(page.read())
    for eachuniversity in soup.findAll('div',{'class':'field-item odd'}):
        print ''.join(eachuniversity.findAll(text=True)).encode('utf-8')
    print ',\n'

edited Oct 19 '13 at 05:02

answered Oct 19 '13 at 04:55

falsetru

357,413
63
732
636

How do I push the output into a .xlx file with comma separated? – Venkateshwaran Selvaraj Oct 19 '13 at 07:08
@Venky, Use [`csv`](http://docs.python.org/2/library/csv.html) module to do that. – falsetru Oct 19 '13 at 07:09
It is becoming hard for me to write it into a csv file because the data are messed up and it is quiet not falling in the exact column as I want. Is there any way I can make it fall in the exact column? – Venkateshwaran Selvaraj Oct 19 '13 at 07:39
@Venky, I thought `.xlx` is similar to `.csv` (becasue of `comma separated`). I don't know about `.xlx`. Sorry. – falsetru Oct 19 '13 at 07:45
Yes. I am trying to write in csv only. In that itself it is messed up – Venkateshwaran Selvaraj Oct 19 '13 at 07:46

ValueError: unsupported format character 'a' (0x61) at index 79

1 Answers1

Reason of the ValueError

Complete code

Reason of the `ValueError`