Is Python GIL needed if extension is single threaded

Question

I am writing a C++ extension for my Python application. I understand that Python GIL is used to prevent multiple thread accessing PyObject at the same time. However, my questions is my extension code is single-threaded, do I still need to acquire GIL?

Let's say in the Python application there are some codes like

import time
from threading import Thread

COUNT = 50000000

def countdown(n):
    while n>0:
        n -= 1

t1 = Thread(target=countdown, args=(COUNT//2,))
t2 = Thread(target=countdown, args=(COUNT//2,))

start = time.time()
t1.start()
t2.start()
t1.join()
t2.join()
end = time.time()

print('Time taken in seconds -', end - start)

I know that even though there are 2 threads, they won't be executed in parallel due to GIL. What if I called my c++ extension in both thread, and my c++ extension is single-threaded? Do I need to consider GIL?

Or another question, I assume that thread will try to hold GIL in python code. If it leaves Python and execute c++ code, will GIL be released?

Possible duplicate of [Concurrency: Are Python extensions written in C/C++ affected by the Global Interpreter Lock?](https://stackoverflow.com/questions/651048/concurrency-are-python-extensions-written-in-c-c-affected-by-the-global-inter) — Iain Shelvington, Aug 05 '19 at 21:09
The short answer is that you can release the GIL in your extension but you need to be careful to re-aquire it — Iain Shelvington, Aug 05 '19 at 21:10

score 1 · Accepted Answer · answered Aug 05 '19 at 21:42

I assume we are talking about CPython and it's GIL.

The implementation guarantees that the GIL is held when the interpreter transfers control to your C++-extension. That means you are free to call back into the interpreter, inspect any object etc; any other thread which would require access to the interpreter is guaranteed to have been blocked while trying to acquire the GIL. This also means that your C++-extension does not need to consider the GIL at all if it does not care about threads which access the interpreter.

The downside of this "safety-first" approach is that the implementation blocks all access to the interpreter while your extension's code is executing. If your extension's code does not need to access the interpreter - e.g. because it can work on local copies/buffers or is about to get blocked on IO - then you should release the GIL immediately, just so that other threads can run, and re-acquire it before returning to the interpreter.

Consider having two Python threads trying to call your extension with fresh work. The first thread will get scheduled, block the interpreter, call your extension and wait for it to return. When it returns, the interpreter switches to the second thread and only then can that thread call your extension. Now it has to wait, blocking thread 1. You effectively have concurrent threads which never run in parallel. Releasing the GIL from within the extension allows the other thread to make progress. It is not required to for the program to be correct though.

Is Python GIL needed if extension is single threaded

1 Answers1