DeadlineExceededError: ApplicationError: 5 in using urllib2.urlopen() function

Question

In my application I am using the urllib2.urlopen() function to call an api and get the result from that api.But this is not working fine.Sometimes it shows the result but sometimes it gives the following error:

Traceback (most recent call last):
  File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/webapp/__init__.py", line 700, in __call__
handler.get(*groups)
  File "/base/data/home/apps/s~malware-app/7.351334968546050391/main.py", line 505, in get
f = urllib2.urlopen('http://whoapi.com/api-v1/?domain=%s&rtype=alexarank&apikey=xyz'% domain)
  File "/base/python_runtime/python_dist/lib/python2.5/urllib2.py", line 124, in urlopen
    return _opener.open(url, data)
  File "/base/python_runtime/python_dist/lib/python2.5/urllib2.py", line 381, in open
response = self._open(req, data)
  File "/base/python_runtime/python_dist/lib/python2.5/urllib2.py", line 399, in _open
  '_open', req)
  File "/base/python_runtime/python_dist/lib/python2.5/urllib2.py", line 360, in _call_chain
result = func(*args)
  File "/base/python_runtime/python_dist/lib/python2.5/urllib2.py", line 1114, in http_open
return self.do_open(httplib.HTTPConnection, req)
  File "/base/python_runtime/python_dist/lib/python2.5/urllib2.py", line 1087, in do_open
r = h.getresponse()
  File "/base/python_runtime/python_dist/lib/python2.5/httplib.py", line 197, in getresponse
self._allow_truncated, self._follow_redirects)
  File "/base/python_runtime/python_lib/versions/1/google/appengine/api/urlfetch.py", line 260, in fetch
return rpc.get_result()
  File "/base/python_runtime/python_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 592, in get_result
return self.__get_result_hook(self)
  File "/base/python_runtime/python_lib/versions/1/google/appengine/api/urlfetch.py", line 364, in _get_fetch_result
raise DeadlineExceededError(str(err))
DeadlineExceededError: ApplicationError: 5

I saw try-except method for this but it dint work for my code. My code block :

 try:                      
   f = urllib2.urlopen('http://whoapi.com/api-v1/?domain=%s&rtype=serverip&apikey=xyzxyz'% domain)
   ip = f.read()
 except DeadlineExceededError, e:
   self.redirect('/error')

I am importing :

from google.appengine.runtime import DeadlineExceededError

From stackoverflow i got that its bcause server didn't responded in the specified time ,and we can handle the exception ..am but not able to do that. Any help would be appreciated. Thank you for your help

possible duplicate of [Unable to handle DeadlineExceededError while using UrlFetch](http://stackoverflow.com/questions/5738146/unable-to-handle-deadlineexceedederror-while-using-urlfetch) — Wooble, Jun 23 '11 at 12:38
Please don't send redirects on an error - just serve the 500 error right then and there. — Nick Johnson, Jun 24 '11 at 00:58

score 12 · Accepted Answer · answered Jun 23 '11 at 12:03

12

The default timeout for URL Fetch requests is just 5 seconds so you might want to increase it by using the urlfetch directly:

from google.appengine.api import urlfetch

try:
    resp = urlfetch.fetch('http://whoapi.com/api-v1/?domain=%s&rtype=serverip&apikey=xyzxyz'% domain, method=urlfetch.GET, deadline=10)
    ip = r.content
except urlfetch.DownloadError:
    self.redirect('/error')

If you consistently find exceeding it anyway, consider using asynchronous requests or moving the logic to task queue.

answered Jun 23 '11 at 12:03

Xion

22,400
10
55
79

thank u all fr ur answers...m making 4 urllib2.urlopen() requests in my one class ...2 of the calls are working fine with urlfetch(){with increased time} but the other 2 are not fetching data with urlfetch method ..so have to use urllib2.urlopen(){any idea why this happened.???}.Any idea to handle the error for urllib2.urlopen() function.The above try-except doesnt worked fr urllib2.urlopen() – bitanalyzer Jun 24 '11 at 11:32

score 0 · Answer 2 · answered Jun 23 '11 at 11:10

0

As you said, the error occurs because you didn't get a response in time, and the request exceeded its deadline.

What you could do is move the request into a task queue, as tasks can run for much longer.

As for catching the exception, have you tried adding a return statement right after self.redirect?

answered Jun 23 '11 at 11:10

Andz

2,228
19
13

no I just redirected it to error class but earlier i tried to print the object e but was not printing anything...(this means that it was not going inside the except block) – bitanalyzer Jun 23 '11 at 11:35

DeadlineExceededError: ApplicationError: 5 in using urllib2.urlopen() function

2 Answers2