0

I have been writing C code that I need to call from Python3. The C file compiles fine and I can import with ctypes CDLL command. Most functions work fine and I can call them from Python. I am using Python version 3.9.16.

However, I am having problems converting and manipulating small integers. I understand that Python3 uses objects for integers between -5 and 256. If I try to convert longs of this size to PyObject in my code the script crashes.

Here's an example function in my C file:

PyObject*
convert_long_to_pyobject(long lng)
{
    return PyLong_FromLong(lng);
}

If I call that from Python with the following, the second call to the function will cause a crash:

from ctypes import CDLL, c_long, py_object
dk = CDLL(my_path)

dk.convert_long_to_pyobject.argtypes = [c_long]
dk.convert_long_to_pyobject.restype = py_object

dk.convert_long_to_pyobject(257) # returns 257
dk.convert_long_to_pyobject(256) # crashes

I haven't written a full C extension. I've just written some functions in a C file, compiled it then loaded with CDLL. I've seen that there is a function called Py_Initialize which maybe sets up the objects for integers in the range (-5, 256) but as far as I can see it is meant for full extensions (and when I tried adding it into a function before a call to PyLong_FromLong it didn't solve my problem.

So my question is, what do I need to do to be able to handle small integers as PyObjects in C? Particularly to convert them between C and Python.

Edit: I was previously running this from Rmarkdown which was hiding the fault returned in Python. The fault returned on the final call to the function within Python is:

Segmentation fault: 11
zhanmusi
  • 161
  • 9
  • 1
    Please add the details of the crash - does the entire Python program crash? Does it throw an exception? – Andrew Henle Feb 01 '23 at 14:42
  • 1
    Also pease update your python version. – Abdul Niyas P M Feb 01 '23 at 14:43
  • I don't think this has anything (directly) to do with caching, but with accurately reflecting the reference count of the object you return. – chepner Feb 01 '23 at 14:53
  • @AndrewHenle I have no details of the crash. I'm actually calling it from an Rmarkdown with `reticulate` and it causes my R session to abort with 'R encountered a fatal error'. I'm new to C and haven't yet figured out how to connect the outputs to Console on my Mac. – zhanmusi Feb 01 '23 at 15:06
  • @AbdulNiyasPM I am using Python 3.9.16. – zhanmusi Feb 01 '23 at 15:06
  • @AndrewHenle I opened Python and ran the code above; the fault when using 256 is: `Segmentation fault: 11` – zhanmusi Feb 01 '23 at 15:15
  • 2
    https://stackoverflow.com/a/51954484/4657412 - you need t use `PyDll` when calling C API functions – DavidW Feb 01 '23 at 15:55
  • @DavidW Yes, thanks, that fixed the problem; for anyone reading this; just change all references from `CDLL` to `PyDLL` in the Python code. – zhanmusi Feb 01 '23 at 16:07
  • You really should show your whole code because I don’t think you’re doing anything you’re supposed to be doing. You don’t need to use python C APIs to call C functions with ctypes – Mark Tolonen Feb 01 '23 at 18:27
  • @MarkTolonen I can't show the code but I need to handle PyObjects such as Python dictionaries after they are passed to a C function. I needed to extract numerical values from a dictionary and increment them, which required a conversion from a PyObject to an int and back to a PyObject, which failed because PyLong_FromLong would error if the int was in the small ints range. – zhanmusi Feb 03 '23 at 00:42
  • In that case, PyDLL has to be used for any function that uses Python C APIs because it doesn’t release the GIL, unlike CDLL and WinDLL. – Mark Tolonen Feb 03 '23 at 01:02

0 Answers0