Is set.copy() atomic in Python?

Question

Suppose we have an instance variable in a class.

class MyClass(object):  
    def __init__():  
        self.set1 = set()

Is the following operation atomic?

set_shallow_copy = self.set1.copy()

I've tried to search around, but the only information I've found is that reading instance variables are atomic.

(Edit) I tried to decompile the bytecode (Python 2.7 on Windows), but it's not too useful. It just shows Python bytecode that calls the copy function on set.

 9 LOAD_FAST                0 (s)  
 12 LOAD_ATTR                1 (copy)  
 15 CALL_FUNCTION            0 
 18 STORE_FAST               1 (s2) 
 21 LOAD_CONST               0 (None) 
 24 RETURN_VALUE

(Edit 2): I wonder if using set2 = set(set1) might be better.

         9 LOAD_GLOBAL              0 (set)  
         12 LOAD_FAST                0 (set1)  
         15 CALL_FUNCTION            1  
         18 STORE_FAST               1 (set2)  
         21 LOAD_CONST               0 (None)  
         24 RETURN_VALUE

I was wondering are we working with python3 or python2.7 here? — DataEngineer, Sep 20 '17 at 23:50
Right now I'm using Python 2.7. Will it be different in Python 3? — Li Chen Koh, Sep 20 '17 at 23:52
Why do you care? Invoking the `copy` method is going to be marginally faster than invoking the built-in `set` constructor, but that has nothing to do with atomicity. What's the goal? — ShadowRanger, Sep 21 '17 at 00:24
I suspect it may depend on what you mean by "atomic". Most operations on builtin objects are "atomic" in the sense that they won't let another operation run in another thread break their internal state. That doesn't mean the operations are guaranteed to run in a specific order, just that the object will be in a sane state at the end. — Blckknght, Sep 21 '17 at 00:27
I see, I think having the object be a sane state at the end would be sufficient. I don't need to guarantee the operations are run in a specific order. — Li Chen Koh, Sep 21 '17 at 00:42

Davis Herring · Accepted Answer · 2020-01-16T03:56:14.573

2

In CPython since version 3.5, the copy is atomic as far as is visible from Python (because, as usual, of the global interpreter lock). No other thread can alter set1 while it is being copied, so you will obtain a copy of some state that the set was in during the manipulations made by (other) threads.

In older versions (like the one tagged here!), the routine that added the elements of the original set to the new (initially empty) copy did not take advantage of the fact that, coming from a set, all the values were unique; as such, it uses == to rediscover that fact. If that comparison is implemented in Python (or by certain C extensions that release the GIL), then the process can be interrupted by other threads (and fail arbitrarily).

edited Jan 16 '20 at 03:56

answered Sep 21 '17 at 04:58

Davis Herring

36,443
4
48
76

Are you sure about that, or is there an authoritative reference for that? I just had a script die in a line `my_set_copy = my_set.copy()` with `RuntimeError: Set changed size during iteration` (the set was indeed modified from a different thread). – bluenote10 Jan 15 '20 at 14:30
@bluenote10: With what Python version? The current implementation doesn’t even need to call a user-defined `__eq__` because it knows the source’s keys are unique. – Davis Herring Jan 15 '20 at 16:22
That was on a legacy system running Python 2.7.12. – bluenote10 Jan 15 '20 at 17:00
1

@bluenote10: Well, that’s the version for which the question was asked, so that’s only fair. I’ve added information for that version, where the answer is **sometimes**. Sorry to have stated it too simply before. – Davis Herring Jan 16 '20 at 03:58
That's very helpful, thanks for digging out the differences. – bluenote10 Jan 16 '20 at 09:48

score -1 · Answer 2 · answered Sep 28 '22 at 16:39

Please use automic_set in shared atomic enterprise

Python atomic for shared data types.

https://sharedatomic.top

The module can be used for atomic operations under multiple processs and multiple threads conditions. High performance python! High concurrency, High performance!

atomic api Example with multiprocessing and multiple threads:

You need the following steps to utilize the module:

create function used by child processes, refer to UIntAPIs, IntAPIs, BytearrayAPIs, StringAPIs, SetAPIs, ListAPIs, in each process, you can create multiple threads.

 def process_run(a):
   def subthread_run(a):
     a.array_sub_and_fetch(b'\x0F')

   threadlist = []
   for t in range(5000):
       threadlist.append(Thread(target=subthread_run, args=(a,)))

   for t in range(5000):
       threadlist[t].start()

   for t in range(5000):
       threadlist[t].join()

create the shared bytearray

a = atomic_bytearray(b'ab', length=7, paddingdirection='r', paddingbytes=b'012', mode='m')

start processes / threads to utilize the shared bytearray

 processlist = []

 for p in range(2):

   processlist.append(Process(target=process_run, args=(a,)))

 for p in range(2):

   processlist[p].start()

 for p in range(2):

   processlist[p].join()

 assert a.value == int.to_bytes(27411031864108609, length=8, byteorder='big')

This and [Does python have atomic CompareAndSet operation?](https://stackoverflow.com/a/73884854) are copy/pastes of your answer on [python atomic data types](https://stackoverflow.com/a/73884815). (except for the first sentence about `automic_set`.) Don't duplicate the same answer in multiple places. — Peter Cordes, Sep 28 '22 at 23:06

Is set.copy() atomic in Python?

2 Answers2

Linked