4

I've read the docs. I've played with examples. But I'm still unable to grasp what exactly asynchronous means when it is useful and where is the magic lots of people seem so crazy about?

If it is only for avoiding to wait for I/O when why to simple run in threads? Why does Deferred needed?

I think I'm missing some fundamental knowledge about computing so those questions. If so what is it?

Moonwalker
  • 2,180
  • 1
  • 29
  • 48
  • 1
    Threads have overhead, and waiting in a thread still cost CPU/Memory. Plus there are synchronization issue. – Kien Truong Mar 08 '12 at 11:12
  • 2
    There is a famous explanation and comparison of different I/O strategies known as [C10K](http://www.kegel.com/c10k.html). – bereal Mar 08 '12 at 11:14
  • 1
    @Dikei and why is this overhead avoided in asynchronous case? You still need to wait and store program data, no? – Moonwalker Mar 08 '12 at 11:18
  • 1
    Dvir Volk's already explained it below :) – Kien Truong Mar 08 '12 at 11:26
  • 1
    [An Introduction to Asynchronous Programming and Twisted](http://krondo.com/?p=1209) – jfs Mar 08 '12 at 15:14
  • Possibly a duplicate? http://stackoverflow.com/questions/80617/asychronous-programming-in-python-twisted/81456#81456 – Glyph Mar 08 '12 at 22:54

1 Answers1

4

like you're five... ok: threads bad, async good!

now, seriously: threads incur overhead - both in locking and switching of the interpreter, and in memory consumption and code complexity. when your program is IO bound, and does a lot of waiting for other services (APIs, databases) to return a response, you're basically waiting on idle, and wasting resources.

the point of async IO is to mitigate the overhead of threads while keeping the concurrency, and keeping your program simple, avoiding deadlocks and reducing complexity.

think for example about a chat server. you have thousands of connections on the server, and you want some people to receive some messages based on which room they are. doing this with threads will be much more complicated than doing this the async way.

re deferred - it's just a way of simplifying your code, instead of giving every function a callback to return to when the operation it's waiting for is ready.

another note - if you want a much simpler and elegant async IO framework, try tornado, which is basically an async web server, with async http client and a replacement for deferred. it's very nicely written and can be used as a general purpose async IO framework.

see http://tornadoweb.org

Not_a_Golfer
  • 47,012
  • 14
  • 126
  • 92
  • 1
    Tornado doesn't have any replacement for Deferred. It has manual callback passing at every call site. That's what *Deferred* is a replacement for. – Jean-Paul Calderone Mar 08 '12 at 16:21
  • 1
    no, tornado has a new library that lets you do that. http://www.tornadoweb.org/documentation/gen.html – Not_a_Golfer Mar 08 '12 at 22:58
  • 1
    Wow... That's almost like http://twistedmatrix.com/documents/current/api/twisted.internet.defer.html#inlineCallbacks but kind of worse. – Jean-Paul Calderone Mar 09 '12 at 02:11
  • 1
    I really don't get the tornado (or cyclone.io) hype. The pythonic hipster way? Deferreds are fine once you get used to them. – ggozad Mar 09 '12 at 05:49
  • 1
    I've never used tornado as just an IO framework, but I can tell you that for building a webserver and using it's http client in apps - it kicks ass. the API is simple and the learning curve is very small, which I can't say about twisted. – Not_a_Golfer Mar 09 '12 at 08:27