gevent: composing greenlets in series

Question

In Twitter's Finagle, map and flatMap can be used to chain multiple asynchronous operations. For example, a simple web scraper might do something like,

downloadURL(url)                     // download a web page
  .flatMap(extractProducts)          // extract individual products for sale on it
  .foreach(saveToDatabase)           // save output to a database

I would like to do something very similar using gevent in Python, but I'm not sure how to do so. Greenlet.link seems to be a start, but it doesn't give me access to the Greenlet containing the final result (as far as I can tell).

How do I mimic Finagle's map and flatMap operations with gevent.Greenlet?

Maybe you should use a Queue instead? – wheaties Oct 29 '13 at 21:39 — wheaties, Oct 29 '13 at 21:39

score 3 · Answer 1 · answered Oct 29 '13 at 21:40

gevent isn't designed for "asynchronous" programming (stacking callbacks, in this context), but for concurrent programming.

The description on gevent's page makes it very clear:

gevent is a coroutine-based Python networking library that uses greenlet to provide a high-level synchronous API on top of the libevent event loop.

What you're looking for is twisted (be warned, though, that's not the most "beginner-friendly" framework there is).

gevent: composing greenlets in series

1 Answers1