I have modeled the following test program on the relevant Python 3.9.2 documentation regarding the synchronization of data among remote processes. As nearly as I can tell, though, it doesn't actually work, so I assume there's something I don't know. The documentation is not explicit about the deployment of SyncManager objects among remote processes, but they are, after all, instances of a subclass of BaseManager, so one must assume that the same technique should work.
After the code below comes shell outputs showing three concurrent invocations that presumably demonstrate the problem I'm having. Although connections are being made to the server, the dict isn't being synchronized. The question is: why?
#!/usr/bin/env python3
# <zteeq.py>
import multiprocessing as mp
import multiprocessing.shared_memory as sm
import multiprocessing.managers as mgrs
import os, sys
#######################################################
class CatalogManager( mgrs.SyncManager): pass
CatalogManager.register( 'get_catalog', dict, mgrs.DictProxy)
#######################################################
class ShareCatalog():
#######################################################
def __init__( self,
catalogManagerAddress,
catalogManagerAuthkey,
**kwargs
):
self.catalogManagerAddress = catalogManagerAddress
self.catalogManagerAuthkey = catalogManagerAuthkey
#######################################################
def start( self):
self.catalogManager = CatalogManager(
self.catalogManagerAddress,
self.catalogManagerAuthkey,
)
try:
self.catalogManager.connect()
print( 'connected self.catalogManager')
except ConnectionRefusedError:
catalogManagerServer = self.catalogManager.get_server()
print( 'starting self.catalogManager')
catalogManagerServer.serve_forever()
self.catalog = self.catalogManager.get_catalog()
###
print( 'pid %d: first stop: %r' % ( os.getpid(), str( self.catalog)))
input()
###
if 'streams' not in self.catalog:
print( 'adding streams')
self.catalog[ 'streams'] = {}
###
print( 'pid %d: second stop: %r' % ( os.getpid(), str( self.catalog)))
input()
###
#######################################################
if __name__ == '__main__':
mp.set_start_method( 'spawn')
shareCatalog = ShareCatalog(
( '127.0.1.1', 43210),
b'abc',
)
shareCatalog.start()
#</zteeq.py>
In the first shell, the SyncManager server starts:
# ./zteeq.py
starting self.catalogManager
Leaving that running, I start the program again in a second shell:
# ./zteeq.py
connected self.catalogManager
pid 2486196: first stop: '{}'
adding streams
pid 2486196: second stop: "{'streams': {}}"
So far, so good. I leave that running and invoke a third time. But the third invocation knows nothing about what the second invocation did; there's no "streams" key in the shared dictionary:
# ./zteeq.py
connected self.catalogManager
pid 2492338: first stop: '{}'
What am I missing?
(Python 3.9.2) (Linux 5.10.0-4-amd64 #1 SMP Debian 5.10.19-1 (2021-03-02) x86_64 GNU/Linux)
Remark: In general, the documentation seems to assume that all SyncManager objects will be created by a shortcut called "multiprocessing.Manager()" which does not provide for the specification of remote socket communications. I assume such objects are intended to be forkishly inherited by all the processes that will use it, as is shown in all the examples I have found so far. But that's not what I'm trying to do.
