Does `psycopg2` in Python 2.x block the GIL

Question

I'm trying to settle an argument with a coworker. Say that I have a Python 2.6 app that uses psycopg2 to communicate with a Postgres database. The app is multithreaded. When a thread makes a database call using psycopg2, does it release the GIL so other threads could also make a database call?

Try [searching for `Py_BEGIN_ALLOW_THREADS` or `PyEval_Save_Thread`](https://github.com/psycopg/psycopg2/search?utf8=%E2%9C%93&q=py_begin_allow_threads+OR+pyeval_save_thread&type=Code) in the source. They're sprinkled here and there, check if your specific scenario applies. — Ilja Everilä, Jun 23 '16 at 09:09

score 3 · Answer 1 · answered Jun 23 '16 at 09:07

3

From a quick glance at the Psycopg release notes, there are many references to releasing the GIL. So apparently it tries to release the GIL when appropriate. You didn't ask about a specific scenario, so I don't think a more specific answer is possible.

answered Jun 23 '16 at 09:07

taleinat

8,441
1
30
44

"A specific scenario" Say just the most common database operation, selecting stuff from a table? – Ram Rachum Jun 23 '16 at 10:04

score 0 · Answer 2 · answered Jun 23 '16 at 09:11

As you said, a rule exists that only the thread that has acquired the GIL may operate on Python objects or call Python/C API functions. In order to emulate concurrency of execution, the Python interpreter regularly tries to switch threads. The lock is also released around potentially blocking I/O operations like reading or writing a file, so that other Python threads can run in the meantime.

Most extension code (psycopg2's code included) manipulating the GIL has the following simple structure:

Save the thread state in a local variable.
Release the global interpreter lock.
... Do some blocking I/O operation ...
Reacquire the global interpreter lock.
Restore the thread state from the local variable.

This means that when a blocking I/O operation (waiting for network response from Postgres, for example) occurs, the GIL is released and other threads can continue their execution. When the blocking I/O operation completes, the thread attempts to acquire the GIL, and continues execution (handling results etc) when it finally does acquire it.

Have a look at psycopg2's implementation here.

Does `psycopg2` in Python 2.x block the GIL

2 Answers2