11

From what I understand, the Global Interpreter Lock allows only a single thread to access the interpreter and execute bytecode. If that's the case, then at any given time, only a single thread will be using the interpreter and its memory.

With that I believe that it is fair to exclude the possibility of having race cases, since no two threads can access the interpreter's memory at the same time, yet I still see warnings about making sure data structures are "thread safe". There is a possibility that it may be covering all implementations of the python interpreter (like cython) which can switch off the GIL and allow true multi threading.

I understand the importance of thread safety in interpreter environments that do not have the GIL enabled. However, for CPython, why is thread safety encouraged when writing multi threaded python code? What is the worse that can happen in the CPython environment?

Anfernee
  • 1,445
  • 1
  • 15
  • 26

2 Answers2

13

Of course race conditions can still take place, because access to datastructures is not atomic.

Say you test for a key being present in a dictionary, then do something to add the key:

if key not in dictionary:
    # calculate new value
    value = elaborate_calculation()
    dictionary[key] = value

The thread can be switched at any point after the not in test has returned true, and another thread will also come to the conclusion that the key isn't there. Now two threads are doing the calculation, and you don't know which one will win.

All that the GIL does is protect Python's internal interpreter state. This doesn't mean that data structures used by Python code itself are now locked and protected.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • I'm just trying to get my head around your example. I mean - in your example - when you write - dictionary[key] = value - then, because the dictionary is a global (shared) hence it is not thread safe? If it were not shared, then it would be thread safe. That is all I asked – variable Nov 06 '19 at 12:42
  • 1
    If it is not shared, then it's not *accessible* for another thread. This hasn't anything to do with locals vs. globals, because locals and globals are just references to an object. The same dictionary can be referenced by a local name and a global name, and it won't make it any more thread safe. – Martijn Pieters Nov 06 '19 at 12:50
1

An important note: the multiprocessing module in Python is synchonous to some degree despite the GIL, in that access to the same variable can occur across different processes simultaneously.

This has a likelyhood of corrupting your data, or at least disrupting your control flow, which would be why thread safety is reccomended.

As to why it happens, despite there only being one interpriter, there isn't anything stopping (at least as far as I can tell) two preinterprited pieces of code accessing the same parts of the shared memory synchonously. When doing say:

import multiprocessing
def my_func ():
    print("hello world")
my_process=multiprocessing.Process (target=my_func, args=(,))
my_process.start ()
my_process.join ()

My understanding is that the time it takes to interprit (in this case) my_func was buried in the overhead it takes to spawn a new process.

In this case, the term "process" is more suitable here, because there are worker threads that are temporarily spawned just to copy data, so there's some data handshaking doing on, so it's actually quite a bit of a different process (pun intended) than the spawning of a traditional thread.

I hope this helps.