2

I am fighting with tornado and the official python oauth2client, gcloud... modules.

These modules accept an alternate http client passed with http=, as long as it has a method called request which can be called by any of these libraries, whenever an http request must be sent to google and/or to renew the access tokens using the refresh tokens.

I have created a simple class which has a self.client = AsyncHttpClient() Then in its request method, returns self.client.fetch(...)

My goal is to be able to yield any of these libraries calls, so that tornado will execute them in asynchronously.

The thing is that they are highly dependant on what the default client - set to httplib2.Http() returns: (response, content)

I am really stuck and cannot find a clean way of making this async

If anyone already found a way, please help.

Thank you in advance

bossylobster
  • 9,993
  • 1
  • 42
  • 61
DevLounge
  • 8,313
  • 3
  • 31
  • 44
  • I have successfully implemented de authentication handler and am storing the tokens into my DB and am setting some secure cookies to know who the user is and if his session is valid. My issue now is not to have to write my own async methods to access google store, google drive, calendar, email... but kind of hack the part of these official libs which really blocks (http requests) – DevLounge Mar 14 '16 at 19:33

1 Answers1

2

These libraries do not support asynchronous. The porting process is not always easy.

oauth2client

Depending on what you want to do maybe Tornado's GoogleOAuth2Mixin or tornado-alf will be enough.

gcloud

Since I am not aware of any Tornado/asyncio implementation of gcloud-python, so you could:

  • you may write it yourself. Again it's not simple transport change of Connection.http or request, all the stuff around must be able to use/yield future/coroutines.

  • wrap it in ThreadPoolExecutor (as @Apero mentioned). This is high level API, so any nested api calls within that yield will be executed in same thread (not using the pool). It could work well.

  • external app (with ProcessPoolExecutor or Popen).

When I had similar problem with AWS couple years ago, I've ended up with executing, asynchronously, CLI (Tornado + subprocess.Popen + some cli (awscli, or boto based)) and simple cases (like S3, basic EC2 operations) with plain AsyncHTTPClient.

kwarunek
  • 12,141
  • 4
  • 43
  • 48
  • 1
    I just found this article, maybe I could implement this: http://blog.trukhanov.net/Running-synchronous-code-on-tornado-asynchronously/ – DevLounge Mar 14 '16 at 20:31
  • Or do some monkey patching of the libs but I'd rather not – DevLounge Mar 14 '16 at 20:36
  • I've edited answer to your point. Of course you have right, ThreadPoolExecutor is an option, it is not time-consuming, might be a bit more difficult to scale on high traffic services. – kwarunek Mar 14 '16 at 21:05