How to read from socket using asyncio add_reader

Question

I have this code:

import sys
import socket
import asyncio

async def main(dest_addr, max_hops=30, timeout=0.5):
    loop = asyncio.get_event_loop()
    queue = asyncio.Queue()

    port = 33434

    rx = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_ICMP)
    rx.settimeout(timeout)
    rx.bind(("", port))

    def reader():
        try:
            _, addr = rx.recvfrom(512)
            addr = addr[0]
        except socket.timeout:
            addr = None
        queue.put_nowait(addr)

    loop.add_reader(rx, reader)

    tx = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)

    for ttl in range(1, max_hops + 1):
        tx.setsockopt(socket.SOL_IP, socket.IP_TTL, ttl)
        tx.sendto(b"", (dest_addr, port))
        addr = await queue.get()
        print(ttl, addr)

    loop.remove_reader(rx)


if __name__ == "__main__":
    dest_name = sys.argv[1]
    dest_addr = socket.gethostbyname(dest_name)
    print(f"traceroute to {dest_name} ({dest_addr})")
    asyncio.get_event_loop().run_until_complete(main(dest_addr))

I am basically trying to implement traceroute using asyncio.

I'm monitoring the socket file descriptor for read availability and invoke reader when I receive data from a device after using the socket.sendto method and wait for the queue to be filled before going to the next step.

However, I my program hangs right after the first iteration, on the second addr = await queue.get().

It seems like the reader callback is only invoked once and never again so the queue is not filled, which is strange because I have a timeout of 0.5s on the rx socket.

tested and got at least 7 hops with addresses, can't reproduce your *hangs right after the first iteration* — RomanPerekhrest, Aug 20 '19 at 11:43
@RomanPerekhrest which address did you use ? (Don't know why it would matter), I ran: `sudo python3.6 traceroute.py www.google.com` and it does hang — NicoAdrian, Aug 20 '19 at 11:52
@RomanPerekhrest How can it be possible ? Whatever happens, the socket timeout should trigger so the queue should be filled after maximum 0.5s — NicoAdrian, Aug 20 '19 at 11:58
I run `sudo python3 ~/projects/drafts/networking/traceroute_emulator.py i.ua` — RomanPerekhrest, Aug 20 '19 at 12:16
@Newlol, when you have a working answer, you can answer your own question instead of providing the solution in the question. — V-R, Aug 20 '19 at 12:42

NicoAdrian · Accepted Answer · 2019-08-20T13:25:43.770

1

Answering my own question:

I think what happens is that, the device (my front router for example) doesn't respond anything so I am never notified when the file descriptor is ready for reading, so the callback is not invoked.

The workaround is to wrap the queue.get() within asyncio.wait_for with a timeout so it doesn't hang forever:

for ttl in range(1, max_hops + 1):
    tx.setsockopt(socket.SOL_IP, socket.IP_TTL, ttl)
    tx.sendto(b"", (dest_addr, port))
    try:
        addr = await asyncio.wait_for(queue.get(), timeout)
    except asyncio.TimeoutError:
        addr = "timeout"
    print(ttl, addr)

edited Aug 20 '19 at 13:25

answered Aug 20 '19 at 12:44

NicoAdrian

946
1
7
18

Worked for me on Ubuntu 18.04, but I had to run the script as root to prevent `File "traceroute.py", line 11, in main rx = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_ICMP) File "/usr/lib/python3.6/socket.py", line 144, in __init__ _socket.socket.__init__(self, family, type, proto, fileno) PermissionError: [Errno 1] Operation not permitted` – V-R Aug 20 '19 at 12:56
1

@V-R Yes because creating an ICMP packet requires a raw socket, which requires root privileges. – NicoAdrian Aug 20 '19 at 13:21

How to read from socket using asyncio add_reader

1 Answers1