Deadline exceeded while posting multiple cards to timelines with a video attached

Question

first of all i'm using python over gae and i'm trying to push a card with a video attachment to all the users that visited my glassware. This is the code:

        #posting video
        media_link = util.get_full_url(self, '/static/video/man_on_the_moon.mp4')
        resp = urlfetch.fetch(media_link, deadline=2000)
        media_video = MediaIoBaseUpload(io.BytesIO(resp.content), mimetype='video/vnd.google-glass.stream-url',
                                            resumable=True)

        users = Credentials.all()
        for user in users:
            creds = StorageByKeyName(Credentials, user.key().name(), 'credentials').get()
            mirror_service = util.create_service('mirror', 'v1', creds)
            timeline_item = {'text': 'video mode v3'}
            mirror_service.timeline().insert(body=timeline_item, media_body=media_video).execute()

On my local machine this code is working flawless but deploying it on gae generates this:

: HTTPException: Deadline exceeded while waiting for HTTP response from URL: https://www.googleapis.com/upload/mirror/v1/timeline?uploadType=resumable&alt=json&upload_id=AEnB2UqeMjtTAaB7wWw-8yuVoaBdxaD5-mkFx2TJo7PynEXmVCkPLlDFo0l_1t8du_JetszdgmHXF9d-VuD8N0XmwXQVBMynow

Any idea on how to solve this? As far as i understood the timeline().insert uploads the media_body each and every time for each user i post the card to. This seems to me quite resource consuming. Right?

Thanks again guys

this is the full stack:

    Deadline exceeded while waiting for HTTP response from URL: https://www.googleapis.com/upload/mirror/v1/timeline?uploadType=multipart&alt=json
Traceback (most recent call last):
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1511, in __call__
    rv = self.handle_exception(request, response, e)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1505, in __call__
    rv = self.router.dispatch(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1253, in default_dispatcher
    return route.handler_adapter(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1077, in __call__
    return handler.dispatch()
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 547, in dispatch
    return self.handle_exception(e, self.app.debug)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 545, in dispatch
    return method(*args, **kwargs)
  File "/base/data/home/apps/s~watchup-stage/1.369179617747477845/main_handler.py", line 340, in post
    mirror_service.timeline().insert(body=timeline_item, media_body=media_video).execute()
  File "lib/oauth2client/util.py", line 128, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "lib/apiclient/http.py", line 676, in execute
    body=self.body, headers=self.headers)
  File "lib/oauth2client/util.py", line 128, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "lib/oauth2client/client.py", line 490, in new_request
    redirections, connection_type)
  File "lib/httplib2/__init__.py", line 1570, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "lib/httplib2/__init__.py", line 1317, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "lib/httplib2/__init__.py", line 1286, in _conn_request
    response = conn.getresponse()
  File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/httplib.py", line 500, in getresponse
    raise HTTPException(str(e))
HTTPException: Deadline exceeded while waiting for HTTP response from URL: https://www.googleapis.com/upload/mirror/v1/timeline?uploadType=multipart&alt=json

This should work, but a little more detail will help troubleshoot. Can you insert other timeline items into Glass from App Engine? Is there anything else in the error log? Can you access the video using an unauthenticated client (e.g. command line cURL or a Chrome incognito window)? — mimming, Jul 25 '13 at 23:27
Hi Jenny, thanks for the reply. I'm able to post images with the same codebase so i guess it's a size related problem. I can definitely browse the file ( https://watchup-stage.appspot.com/static/video/man_on_the_moon.mp4 ). — FaustoDassenno, Jul 26 '13 at 08:10
More log: response = conn.getresponse() : File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/httplib.py", line 500, in getresponse : raise HTTPException(str(e)) : HTTPException: Deadline exceeded while waiting for HTTP response from URL: https://www.googleapis.com/upload/mirror/v1/timeline?uploadType=resumable&alt=json&upload_id=AEnB2UpTMeODyvX9ovlkinXWHUlb_0GLTH-sWpgE1_isnpt0jsJTh-myGIQgCLf4je9A2BCAHcrkQYJIHks29PtV_a33bKj7QA — FaustoDassenno, Jul 26 '13 at 08:15

Jessie A. Morris · Accepted Answer · 2013-08-01T02:28:13.583

I too was experiencing this type of issue. In order to solve it, I had to increase the timeout for HTTP requests as well as move to a Queue.

I submitted a pull request to the Google Mirror API Python example, but it got rejected to keep things "simple".

You can checkout my diff at https://github.com/JessieAMorris/mirror-quickstart-python/commit/b67c0ecdf83207b0c3b596173c95a53804b23525.

Basically, the synopsis is:

Move processing to a queue
import urlfetch and httplib2

Increase the timeouts for urlfetch and httplib2 like so:

urlfetch.set_default_fetch_deadline(45)
httplib2.Http(timeout=45)

With those changes in place, I don't get timeouts any more.

score 1 · Answer 2 · answered Jul 28 '13 at 19:38

1

You have 60 seconds to complete a front-end request on Google App Engine. Locally however, that restriction doesn't seem to stick.

Your only options for longer processing are task queues (10 minute deadline) and backends (indefinite).

answered Jul 28 '13 at 19:38

Tombatron

1,567
1
13
27

Ok, thanks for your suggestion. I tried moving the logic to a queue. In local everything is working as a charm but in prod i got this 15 seconds after the job starded: 2013-07-31 11:27:15.725 Deadline exceeded while waiting for HTTP response from URL: https://www.googleapis.com/upload/mirror/v1/timeline?uploadType=resumable&alt=json&upload_ – FaustoDassenno Jul 31 '13 at 10:28
Interesting. Can you share the entire call stack? Is MediaIoBaseUpload actually uploading a file? – Tombatron Jul 31 '13 at 12:09
I was mistaken, the behavior is not consistent. Now the error is back.HTTPException: Deadline exceeded while waiting for HTTP response from URL: https://www.googleapis.com/upload/mirror/v1/timeline?uploadType=multipart&alt=json And the handler is called from a queue. – FaustoDassenno Jul 31 '13 at 20:24
Am I just not seeing this correctly or is it possible that the request that is timing out is originating in the httplib2 that you have referenced? If that's the case, this is probably just a matter of that deadline you are setting not "trickling" down. – Tombatron Jul 31 '13 at 23:41

Deadline exceeded while posting multiple cards to timelines with a video attached

2 Answers2

Linked