Adding values to set contained in Multiprocessing.Manager().list()

Question

I am trying to update a Manager().list() of sets with tuples but having trouble getting anything to add into the set. You can replicate the issue with the below code:

from multiprocessing import Manager

l = Manager().list()
for _ in range(10):
    l.append(set())
l[0].add((10,10)
# returns None
l[0] = {(10,10)}
# sets l[0] to {(10,10}
l[0].add((11,11))
# returns None
l[0].add(10)
# returns None

It seems that for some reason the add method is returning None no matter what. I can't believe this is the expected behavior but perhaps I am just missing something here. In the end all I want to be able to do is:

l[i].add(some_tuple)

And have it properly store the value.

Answer 1

The answer is buried in a Note section here (this is the Python 2.7 documentation but the same applies in Python 3.x):

Note Modifications to mutable values or items in dict and list proxies will not be propagated through the manager, because the proxy has no way of knowing when its values or items are modified. To modify such an item, you can re-assign the modified object to the container proxy:

# create a list proxy and append a mutable object (a dictionary)
lproxy = manager.list()
lproxy.append({})
# now mutate the dictionary
d = lproxy[0]
d['a'] = 1
d['b'] = 2
# at this point, the changes to d are not yet synced, but by
# reassigning the dictionary, the proxy is notified of the change
lproxy[0] = d

In your case, once your list-proxy object l has ten sets inside it, you can (in any independent process—note that you cannot do this in two processes simultaneously as these are independent objects too!) do this:

x = l[0]
x.append((10, 10))
l[0] = x

or, if you like:

l[0] = l[0] | set([(10, 10)])

or of course the newfangled Python 3 set syntax:

l[0] = l[0] | {(10, 10)}

---

Beware: multiprocessing.managers.ListProxy does not (well, cannot, really) implement __ior__ directly on a selected object. Although you can write these using l[0] |= ... if you prefer:

l[0] |= ...

Python actually implements this as:

1. Fetch the value from l[index] (with locking and/or proxying as needed), then unlock the managed object, returning just the temporary.
2. Perform the operation on the temporary.
3. Store the result to l[index] (with locking and/or proxying just like before).

This is why you must be careful not to modify, e.g., l[0] from two different processes at the "same time": one of them will assign first, then the other will assign second, discarding the first-assigned value.

Answer 2

pardon if i'm not specific with answer, but try this instead

l.insert(index, 'whatever you want to add (tuple maybe)')

index is the place in which you want to add, cheers