Python multiprocessing Manger OSError "Only one usage of each socket address"

Question

I am creating a communication platform in python (3.4.4) and using the multiprocessing.managers.BaseManager class. I have isolated the problem to the code below.

The intention is to have a ROVManager(role='server') instance running in one process on the main computer and providing read/write capabilities to the system dictionary for multiple ROVManager(role='client') instances running on the same computer and a ROV (remotely operated vehicle) connected to the same network. This way, multiple clients/processes can do different tasks like reading sensor values, moving motors, printing, logging etc, all using the same dictionary. start_reader() below is one of those clients.

Code

from multiprocessing.managers import BaseManager
import multiprocessing as mp
import sys    

class ROVManager(BaseManager):
    def __init__(self, role, address, port, authkey=b'abc'):
        super(ROVManager, self).__init__(address=(address, port),
                                         authkey=authkey)
        if role is 'server':
            self.system = {'shutdown': False}
            self.register('system', callable=lambda: self.system)
            server = self.get_server()
            server.serve_forever()
        elif role is 'client':
            self.register('system')
            self.connect()    

def start_server(server_ip, port_var):
    print('starting server')
    ROVManager(role='server', address=server_ip, port=port_var)    

def start_reader(server_ip, port_var):
    print('starting reader')
    mgr = ROVManager(role='client', address=server_ip, port=port_var)
    i = 0
    while not mgr.system().get('shutdown'):
        sys.stdout.write('\rTotal while loops: {}'.format(i))
        i += 1    

if __name__ == '__main__':
    server_p = mp.Process(target=start_server, args=('0.0.0.0', 5050))
    reader_p = mp.Process(target=start_reader, args=('127.0.0.1', 5050))
    server_p.start()
    reader_p.start()
    while True:
        # Check system status, restart processes etc here
        pass

Error

This results in the following output and error:

starting server
starting reader
Total while loops: 15151
Process Process - 2:
Traceback(most recent call last):
    File "c:\python34\Lib\multiprocessing\process.py", line 254, in _bootstrap
        self.run()
    File "c:\python34\Lib\multiprocessing\process.py", line 93, in run 
        self._target(*self._args, **self._kwargs)
    File "C:\git\eduROV\error_test.py", line 29, in start_reader
        while not mgr.system().get('shutdown'):
    File "c:\python34\Lib\multiprocessing\managers.py", line 640, in temp
        token, exp = self._create(typeid, *args, **kwds)
    File "c:\python34\Lib\multiprocessing\managers.py", line 532, in _create
        conn = self._Client(self._address, authkey=self._authkey)
    File "c:\python34\Lib\multiprocessing\connection.py", line 496, in Client
        c = SocketClient(address)
    File "c:\python34\Lib\multiprocessing\connection.py", line 629, in SocketClient
        s.connect(address)
OSError: [WinError 10048] Only one usage of each socket address(protocol / network address / port) is normally permitted

My research

Total while loops are usually in the range 15000-16000. From my understanding it seems like a socket is created and terminated each time mgr.system().get('shutdown') is called. Windows then runs out of available sockets. I can't seem to find a way to set socket.SO_REUSEADDR.

Is there a way of solving this, or isn't Managers made for this kind of communication? Thanks :)

Answer 1

As error suggests Only one usage of each socket address in general , you can/should bind only a single process to a socket ( unless you design your application accordingly, by passing SO_REUSEADDR option while creating socket) . These lines

server_p = mp.Process(target=start_server, args=('0.0.0.0', 5050))
reader_p = mp.Process(target=start_reader, args=('127.0.0.1', 5050))

creates two processes on same port 5050 & so the error.
You can refer here to learn how to use SO_REUSEADDR & its implications but i am quoting the main part which should get you going

The second socket calls setsockopt with the optname parameter set to SO_REUSEADDR and the optval parameter set to a boolean value of TRUE before calling bind on the same port as the original socket. Once the second socket has successfully bound, the behavior for all sockets bound to that port is indeterminate. For example, if all of the sockets on the same port provide TCP service, any incoming TCP connection requests over the port cannot be guaranteed to be handled by the correct socket — the behavior is non-deterministic.