0

I have a relatively simple script I use to run map jobs over my data, and I'd like it to run a faster by using gevent to make reads from the datastore non-blocking.

I tried adding gevent.monkey.patch_socket() to the top of remote_api_shell.py, but after I log in, it throws a urllib2.URLError (traceback).

I'm new to both GAE and gevent. Is there an easier way to accomplish what I'm trying to do? Could someone point me in the right direction to get this working?

Andrey Fedorov
  • 9,148
  • 20
  • 67
  • 99

1 Answers1

1

If using ndb:

You can use ndb with *_async datastore operations:
https://developers.google.com/appengine/docs/python/ndb/async

For example, to retrieve a list of elements by key:

ndb.get_multi_async(list_of_keys)

The great thing with ndb is that every datastore operation on keys or entities has an async equivalent.

If using db:

You can use the Async Datastore API.

For example, to retrieve a list of elements by key:

from google.appengine.ext import db
db.get_async(list_of_keys)

Where list_of_keys contains elements that are either strings, instances of google.appengine.api.datastore_types.Key, or instances of google.appengine.api.datastore.Entity.

Similarly you can use db.put_async on lists of entities and db.delete_async on lists similar to those for db.get_async.

bossylobster
  • 9,993
  • 1
  • 42
  • 61
  • This is certainly an option, but I'd rather find a solution that doesn't involve migrating all of my model classes to ndb.... – Andrey Fedorov Dec 20 '12 at 03:27
  • See the update. Though I implore you, please, **please**, **PLEASE** take the time to check out `ndb`. It is head and shoulders a better datastore API and gives you caching that you don't have to think about right out of the box. – bossylobster Dec 20 '12 at 16:40
  • Cool, I'll definitely check it out, thanks for the tip, and the thorough answer, as well! – Andrey Fedorov Dec 20 '12 at 18:44