0

I've been experiencing a memory leak when calling some C code I wrote from Python 3.10.6 on Ubuntu 22.04.1 LTS.

https://github.com/seung-lab/mapbuffer/blob/main/mapbufferaccel.c#L95-L110

When used in isolation, there is no memory leak, but the following code appears to leak:

https://github.com/seung-lab/igneous/blob/master/igneous/tasks/mesh/multires.py#L390-L406

When I replace the C call with a python implementation, the leak disappears:

https://github.com/seung-lab/mapbuffer/blob/main/mapbuffer/mapbuffer.py#L197

This only happens on Linux, on MacOS there is no leak. I tried calling both the python and C versions at the same time and using the python result while deleting elements from the C version. I got the C version only returning -1 and it seems simply calling the C version causes the leak on Linux.

Any ideas what's going on? Is it some part of glibc that is causing a problem? Did I forget to call some cleanup function in the Python C API?

Here is the non-leaking Python implementation of that function:

    def eytzinger_search(target, arr):
        mid = 0
        N = len(arr)

        while mid < N:
            if arr[mid] == target:
                return mid
            mid = mid * 2 + 1 + int(arr[mid] < target)

        return -1

    k = eytzinger_search(np.uint64(label), index[:, 0])

Thanks so much!

SapphireSun
  • 9,170
  • 11
  • 46
  • 59
  • 1
    I don't see what your second code snippet has to to with calling eytzinger_search. Am I missing something? But I don't see anything obviously wrong with the C code. Is there a reason you're using the buffer interface rather than the PyArray interface? – Frank Yellin Jun 27 '23 at 02:33
  • How to reproduce the problem? – Tom Jun 27 '23 at 02:56

1 Answers1

0

It turns out that the "y*" in the below code was allocating a buffer that needed to be cleared.

if (!PyArg_ParseTuple(args, "ny*", &label, &index)) {
    return NULL;
}

Adding the following code fixed it!

PyBuffer_Release(&index);
SapphireSun
  • 9,170
  • 11
  • 46
  • 59
  • 1
    My eyes must be failing me. I checked for the `PyBuffer_Release`, and thought I saw it. But again, if the argument is just an array, is there a reason you're not using PyArray_DATA(arg) to just grab the array's memory. You can also easily grab the number of elements in the array, the total size, etc. – Frank Yellin Jun 28 '23 at 19:45
  • You weren't wrong! I corrected the code before I answered the SO. Sorry about that! Thank you for the tip about PyArray_DATA, that's a good tip. I was wondering why I needed to make a copy in the first place! The reason I was using PyBuffer instead of PyArray is mainly due to inexperience with the C API. – SapphireSun Jun 29 '23 at 02:41