5

the normal behavior of urllib/urllib2 is if an error code is sent in the header of the response (i.e 404) an Exception is raised.

How do you look for specific errors i.e (40x, or 50x) based on the different errors, do different things. Also, how do you read the actual data being returned HTML/JSON etc (The data usually has error details which is different to the HTML error code)

ismail
  • 3,882
  • 5
  • 36
  • 47

2 Answers2

9

urllib2 raises a HTTPError when HTTP errors happen. You can get to the response code using code on the exception object. You can get the response data using read():

>>> req = urllib2.Request('http://www.python.org/fish.html')
>>> try:
>>>     urllib2.urlopen(req)
>>> except urllib2.HTTPError, e:
>>>     print e.code
>>>     print e.read()
>>>
404
<actual data response will be here>
Dominic Rodger
  • 97,747
  • 36
  • 197
  • 212
  • Dominic, the acutal data is not returned, i have updated the question with the exception code. – ismail Nov 26 '09 at 13:54
  • This will fail in a lot of cases since `URLError` doesn't have `code` attribute and `read()` method itself, but only its subclass `HTTPError` has. – Denis Otkidach Nov 26 '09 at 13:57
  • @Issy - really? It works for me. What's not working about it? Your second comment suggested it was working! If mine's not working and @Denis' is then you should accept his answer instead. – Dominic Rodger Nov 26 '09 at 14:38
  • In fact it works since HTTPError is subclass of URLError, so catching URLError will catch HTTPError too. But it contained a hidden error which might break in some other cases. – Denis Otkidach Nov 26 '09 at 15:01
1

In urllib2 HTTPError exception is also a valid HTTP response, so you can treat an HTTP error as an exceptional event or valid response. But in urllib you have to subclass URLopener and define http_error_<code> method[s] or redefine http_error_default to handle them all.

Denis Otkidach
  • 32,032
  • 8
  • 79
  • 100