Exception: access violation reading when calling a C function from Python

Question

I wanted to play around a bit with the Python C Api. But ran into an error.

OSError: exception: access violation writing 0x0000000000000020

The error occurs on the lines PyObject_RichCompare(first, second, Py_LT)

There are no errors in the first and second variables. If you remove the line PyObject_RichCompare(first, second, Py_LT) everything works.

Tried building "DLL" both on Linux in GCC and on Windows in Visual Studio. Everywhere I get this error

C code

#include "Python.h"

extern "C" {
    __declspec(dllexport)
    long test(PyObject* list) {
        PyObject* first, * second;
        int t = PyList_Size(list);
        first = PyList_GetItem(list, 0);
        second = PyList_GetItem(list, 1);
        PyObject* result = PyObject_RichCompare(first, second, Py_LT);
        return PyLong_AsLong(first) + PyLong_AsLong(second);
    }
}

And Python Code

import ctypes

lib = ctypes.CDLL('please.dll')
lib.test.restype = ctypes.c_long
lib.test.argtypes = [ctypes.py_object]

py_values = [1, 645, 546, 8646, 45646, 6545688, 5465]
a = lib.test(py_values)

print(a)

When you are calling code that uses the Python C API, you *must* load it as [a `PyDLL`](https://docs.python.org/3/library/ctypes.html#ctypes.PyDLL), not `CDLL`. As a `CDLL` it releases the GIL, which makes basically any interaction with any Python level object inherently unsafe. Probably not the cause of your problem, but things will eventually explode if a thread is introduced into your program (even if this library is only used by a single thread, reference counts can get out of sync due to the GIL holding thread and the thread calling this library racing). — ShadowRanger, Sep 15 '22 at 23:35
Why is the broken line there at all? `PyObject* result = PyObject_RichCompare(first, second, Py_LT);` leaks a reference to a `bool` and don't even use the result... — ShadowRanger, Sep 15 '22 at 23:37

score 2 · Accepted Answer · answered Sep 15 '22 at 23:35

2

The GIL must be held to use Python C APIs. Use PyDLL for that:

lib = ctypes.PyDLL('please.dll')

answered Sep 15 '22 at 23:35

Mark Tolonen

166,664
26
169
251

While this is true, I don't think it will change anything in a single-threaded program, the problem is likely not related to it (though they should absolutely switch to using `PyDLL` as well). – ShadowRanger Sep 15 '22 at 23:36
@ShadowRanger I tested the change. The result prints correctly (646, the addition of the first two elements of the list). – Mark Tolonen Sep 15 '22 at 23:39
Awesome, that's enough for me. I knew it was bad form, but I guess the mechanics of going through all the rich comparison code paths eventually do stuff that trigger bugs when they're hit without the GIL. – ShadowRanger Sep 15 '22 at 23:39
@ShadowRanger And just for completeness I'll note that I reproduced the crash with `CDLL`. Undefined behavior is undefined behavior after all – Mark Tolonen Sep 15 '22 at 23:53

Exception: access violation reading when calling a C function from Python

1 Answers1