Get an asyncio server to call another server

Question

So I have modified the example of asyncio client and server (found here: https://docs.python.org/3/library/asyncio-protocol.html#protocol-example-tcp-echo-server-and-client) and all I want is for client.py to call serverone.py, and in turn for it to call servertwo.py.

client.py

#!/usr/bin/env python3.4
import asyncio

class EchoClient(asyncio.Protocol):
    message = 'This is the Client'

    def connection_made(self, transport):
        transport.write(self.message.encode())

    def data_received(self, data):
        print('data received: {}'.format(data.decode()))

    def connection_lost(self, exc):
        asyncio.get_event_loop().stop()

loop = asyncio.get_event_loop()
coro = loop.create_connection(EchoClient, '127.0.0.1', 8888)
loop.run_until_complete(coro)
loop.run_forever()
loop.close()

serverone.py

#!/usr/bin/env python3.4

import asyncio

class EchoClient(asyncio.Protocol):
    message = 'Server One sending message'

    def connection_made(self, transport):
        transport.write(self.message.encode())

    def data_received(self, data):
        print('data received: {}'.format(data.decode()))

    def connection_lost(self, exc):
        asyncio.get_event_loop().stop()

class EchoServer(asyncio.Protocol):
    def connection_made(self, transport):
        peername = transport.get_extra_info('peername')
        self.transport = transport

    def data_received(self, data):
        loop = asyncio.get_event_loop()
        coro = loop.create_connection(EchoClient, '127.0.0.1', 8889)
        loop.run_until_complete(coro)
        # close the socket
        self.transport.close()
        loop.close()

loop = asyncio.get_event_loop()
coro = loop.create_server(EchoServer, '127.0.0.1', 8888)
server = loop.run_until_complete(coro)
try:
    loop.run_forever()
except KeyboardInterrupt:
    print("exit")
finally:
    server.close()
    loop.close()

servertwo.py

#!/usr/bin/env python3.4

import asyncio

class EchoServer(asyncio.Protocol):
    def connection_made(self, transport):
        peername = transport.get_extra_info('peername')
        self.transport = transport

    def data_received(self, data):
        print('data received: {}'.format(data.decode()))
        self.transport.write(data)
        # close the socket
        self.transport.close()

loop = asyncio.get_event_loop()
coro = loop.create_server(EchoServer, '127.0.0.1', 8889)
server = loop.run_until_complete(coro)
try:
    loop.run_forever()
except KeyboardInterrupt:
    print("exit")
finally:
    server.close()
    loop.close()

I launch servertwo.py and serverone.py in a terminal, then call client.py. Things partially work; the client does call serverone which does call servertwo, but then serverone fails with this error:

Exception in callback <bound method _SelectorSocketTransport._read_ready of <asyncio.selector_events._SelectorSocketTransport object at 0x7fbf4453b048>>()
handle: Handle(<bound method _SelectorSocketTransport._read_ready of <asyncio.selector_events._SelectorSocketTransport object at 0x7fbf4453b048>>, ())
Traceback (most recent call last):
  File "/usr/lib64/python3.4/asyncio/events.py", line 39, in _run
    self._callback(*self._args)
  File "/usr/lib64/python3.4/asyncio/selector_events.py", line 458, in _read_ready
    self._protocol.data_received(data)
  File "./serverone.py", line 25, in data_received
    loop.run_until_complete(coro)
  File "/usr/lib64/python3.4/asyncio/base_events.py", line 203, in run_until_complete
    self.run_forever()
  File "/usr/lib64/python3.4/asyncio/base_events.py", line 179, in run_forever
    raise RuntimeError('Event loop is running.')
RuntimeError: Event loop is running.

The documentation doesn't cover a lot of odd use cases, so I'm a bit stuck. Should I be using asyncio.async to make the call? Am I approaching the problem correctly?

How do I fix or avoid the RuntimeError?

I'd hardly call this an "odd" use case. Writing a server which needs to make outbound asynchronous calls in the course of serving an inbound request seems pretty usual to me. Came here looking for examples on how to do exactly that. — DanielSank, Jun 27 '16 at 22:42

score 3 · Accepted Answer · answered Sep 03 '14 at 17:47

3

You can use asyncio.async to schedule the coroutine returned by create_connection to be run by the event loop, and then use the add_done_callback method of the asyncio.Future (more specifically, an asyncio.Task) that async returns to shut down the loop once the coroutine is done:

class EchoServer(asyncio.Protocol):
    def connection_made(self, transport):
        peername = transport.get_extra_info('peername')
        self.transport = transport

    def data_received(self, data):
        loop = asyncio.get_event_loop()
        coro = loop.create_connection(EchoClient, '127.0.0.1', 8890)
        fut = asyncio.async(coro)
        fut.add_done_callback(self.shutdown)

    def shutdown(self, *args):
        self.transport.close()
        loop.stop()

answered Sep 03 '14 at 17:47

dano

91,354
19
222
219

That answer definitely gives me a leg up, but even without the `shutdown()` method, the server will stop once it runs the `async` call. I've tried adding `run_forever()` in the `data_recieved()` method but to no avail. I can't have the server stopping after it makes a call to the other server. But +1 for a call that doesn't end in an exception. – NuclearPeon Sep 03 '14 at 19:03
I should explain. The question I asked was a simplified problem of what I was facing. My real problem was that I needed to communicate between multiple daemons (already forked). By adding in your solution, the daemons keep running and I can call the server multiple times without the loop dying on me, so that does solve my problem. However, in my example, `serverone.py` still exits its loop. – NuclearPeon Sep 03 '14 at 19:09
1

@NuclearPeon That's because you stop the event loop in `EchoClient.connection_lost`. Don't do that! :) – dano Sep 03 '14 at 19:11
Aha! I also removed the `loop.stop()` in `shutdown()` and it all works as expected! Thank you so much @dano! – NuclearPeon Sep 03 '14 at 19:12

score 0 · Answer 2 · answered Nov 01 '19 at 03:57

0

Replace asyncio.async with asyncio.ensure_future and it'll run with the upvoted answer.

answered Nov 01 '19 at 03:57

Roy

9
1

Get an asyncio server to call another server

2 Answers2