5

I'm going through some asyncio source for networking and implementation raised a question in my head.

To create a non-blocking I/O when waiting for the data to arrive from a socket, asyncio.StreamReader.read() at its end calls _wait_for_data method that creates an empty Future and awaits it.

That future is set as finished (which allows it to be finally awaited) in _wakeup_waiter method that is called when a new data arrives to the stream (feed_data method).

That makes complete sense.

My question is:

Why not use asyncio.Event? It feels to me like Event was designed exactly for these purposes. In fact, you wouldn't have to create a new Future on each _wait_for_data call, but would initialize a single Event in a class and would simply toggle its value during its lifetime. It also has specific .wait() method for awaiting its value to become True (for when the new data arrives from the socket).

Can anyone elaborate if there's an actual difference between the two approaches? Or is it just a method chosen arbitrarily?

Radek Kysely
  • 100
  • 1
  • 7

2 Answers2

4

While usually you can replace Future with Event if you don't care about data future will be populated with, I don't think it's true in this case.

In the code self._waiter is used not only to indicate wakeup event, but also to indicate exception that happened. set_exception to Future means that exact this exception will be raised inside code that awaits for future:

# 

waiter.set_exception(exc)  # Exception set here...

# 

self._waiter = self._loop.create_future()
try:
    yield from self._waiter  # ... will be raised here and propagated to outer code
finally:
    self._waiter = None

You wouldn't be able to achieve this if you change self._waiter to an Event.

Mikhail Gerasimov
  • 36,989
  • 16
  • 116
  • 159
1

Future is low level primitive. It is powerful but user code usually don't need it. Like Linux has futex but user code uses high level objects like lock and recursive lock.

Andrew Svetlov
  • 16,730
  • 8
  • 66
  • 69
  • Thanks for adding this, Andrew. Do you have any resources on hand on how efficient creating new Futures is? – Radek Kysely Feb 17 '18 at 18:16
  • What do you mean by efficiency? It is written in C and everything in asyncio is based on top of futures. Another question is: should you use the future? I saw too many errors in future-based user code. – Andrew Svetlov Feb 17 '18 at 18:33
  • I was worried about the performance since in this case a new Future has to be created quite often—but I guess if it's so low-level, implemented in C and is used under whole asyncio, it souldn't really be an issue. Do you have any tips on how to avoid those errors you've seen? Read the docs carefully, I guess? – Radek Kysely Feb 18 '18 at 03:07
  • Usually everything works just fine in case of normal flow but fails/hangs/leaks under some conditions. See https://bugs.python.org/issue32734 https://bugs.python.org/issue32841 https://bugs.python.org/issue32574 – Andrew Svetlov Feb 18 '18 at 11:35
  • Will check those out. Thank you – Radek Kysely Feb 19 '18 at 13:05