In Python, yielding from lock in DatagramProtocol.datagram_received makes the function never called

Question

I want to synchronize data from between coroutines and I end up with a method not being called whenever there is "yield" inside it.

To be more precise, when I implement a DatagramProtocol class with the method datagram_received as per the doc (inspired from this), everything works fine, I receive the data. As soon as I add a "yield" inside the method datagram_received, the method is never called anymore. Here is an example:

loop = asyncio.get_event_loop()
lock = asyncio.Lock(loop=loop)

class MyProtocol(asyncio.DatagramProtocol):
    def datagram_received(self, data, addr):
        global my_data, lock
        print("here")
        # uncomment the following lines and datagram_received is 
        # not called at all (never see the "here" on the console)
        #yield from lock
        #try:
        #    my_data = float(data.decode())
        #finally:
        #    lock.release()

loop.run_until_complete(loop.create_datagram_endpoint(MyProtocol, sock=create_socket(10000)))

loop.run_forever()

How can a method suddenly get not being called depending on the content of the method?

What am I missing? How the synchronization should be done?

`yield` caused your function to return a generator. You need to iterate over it for the code to execute. — dirn, Nov 20 '17 at 18:16
@dim, could you put that as an answer? with maybe an example? — Philippe Cayouette, Nov 21 '17 at 02:51

score 1 · Accepted Answer · edited Jun 20 '20 at 09:12

What am I missing?

Documentation that inspired you also states:

Coroutines can be scheduled in a protocol method using ensure_future(), but there is no guarantee made about the execution order. Protocols are not aware of coroutines created in protocol methods and so will not wait for them.

To have a reliable execution order, use stream objects in a coroutine with yield from. For example, the StreamWriter.drain() coroutine can be used to wait until the write buffer is flushed.

You cannot yield from/await inside datagram_received, you may:

class MyProtocol(asyncio.DatagramProtocol):
    def datagram_received(self, data, addr):
        global my_data, lock
        print("here")
        loop.ensure_future(some_function())
        
    @asyncio.coroutine
    def some_function(self):
        yield from lock
        try:
            my_data = float(data.decode())
        finally:
            lock.release()

How can a method suddenly get not being called depending on the content of the method?

Use of yield or yield from in a function, makes it a generator. So datagram_received returns generator object. To actually execute the code (till yield) you should use next, asyncio does it with (generator-based) coroutines (but again datagram_received isn`t one)

>>> def test():
...     print('test')
...     yield from 'A'
... 
>>> test()
<generator object test at 0x7f4165d42fc0>
>>> next(test())
test
'A'

More about generators: https://jeffknupp.com/blog/2013/04/07/improve-your-python-yield-and-generators-explained/

Awesome, hope it helps other programmers that come from other languages like me! — Philippe Cayouette, Nov 26 '17 at 18:34

In Python, yielding from lock in DatagramProtocol.datagram_received makes the function never called

1 Answers1

What am I missing?

How can a method suddenly get not being called depending on the content of the method?