Unit test for memory leaks in Python

Question

Is there a way to run a unit test for memory leaks in Python? I've written some C++ binded code and want to make sure it doesn't leak memory.

For example, I would like a test like the following:

memory_usage_0 = 0
memory_usage_1 = 0
memory_usage_0 = get_memory_usage()
do_cpp_bound_operation()
memory_usage_1 = get_memory_usage()
assert memory_usage_0 == memory_usage_1

you do not have to worry about memory leaks in Python, the memory is managed — L.Grozinger, Jul 20 '20 at 19:40
I've created a C extension though which I want to make sure is reference counting (as in https://docs.python.org/3/c-api/refcounting.html) correctly. — Daishisan, Jul 20 '20 at 19:42
I don't know much about valgrind, but it doesn't seem like it's easily made into a unittest AFAICT. — Daishisan, Jul 20 '20 at 20:00
The `gevent` library has some unittesting leak checks https://github.com/gevent/gevent/blob/4152f909a8bd431c3f62f96708229f02a19d6e96/src/gevent/testing/leakcheck.py. I don't 100% know how they're doing it, but it might be worth a look — DavidW, Jul 21 '20 at 07:06
@Daishisan is it possible to share how exactly you solved this ? — stonelazy, Feb 28 '22 at 17:26

score 1 · Answer 1 · answered Jul 20 '20 at 23:35

If you are willing for your unit test to be slow, here is approach to try based on your coding sample:

Change your code fragment to look like this:

gc.collect()
callCppCoreGenerationFunction()
do_cpp_bound_operation()
gc.collect()
callCppCoreGenerationFunction()
compareCores()

Your callCppCoreGenerationFunction can call a c or C++ function that is as simple as forking a new process and having the child kill itself, for example, by having the child reference a null pointer. It is key that this be simple, because you don't want this function to muddy the waters by doing extra python allocations during the generation of the cores.

The key part to discuss here is compareCores(). For this you can use the standard python code that forks processes because you already have cores to compare and don't need to worry about distorting the results. You can, for example fork a shell script to calculate your answer.

So now the question is what should go in your shell script. The first thing it should do is list the cores in order of creation so that you can compare the two most recent ones. Then you open each core in chap (open source software available at https://github.com/vmware/chap). For each core you should open it in chap and run the following commands from the chap prompt:

count leaked

That looks something like this and will catch leaked natively allocated objects and a few, but definitely not all, leaked python objects.

chap> count leaked
0 allocations use 0x0 (0) bytes.

That particular command is quite reliable because chap is designed to avoid false positives on the leaks. So you could even run this just on the second of the two cores and the first core would not be needed at all for such a check. If you do find leaks are other chap commands you could use to analyze the leaks but if you only want to know whether you have such leaks, and not why, counting is enough.

describe used

That will describe all allocations (python and native) that have been allocated and not freed. The output of that looks something like this, but with many more lines:

Anchored allocation at 7f505b117630 of size 40
This allocation matches pattern ContainerPythonObject.
This has a PyGC_Head at the start so the real PyObject is at offset 0x10.
This has reference count 1 and python type 0x903f20 (dict)

Anchored allocation at 7f505b117670 of size 40
This allocation matches pattern ContainerPythonObject.
This has a PyGC_Head at the start so the real PyObject is at offset 0x10.
This has reference count 1 and python type 0x8fd660 (list)

Anchored allocation at 16f8a80 of size 238
This allocation matches pattern SimplePythonObject.
This has reference count 1 and python type 0x7f5e824cdfe0 (str)
This has a string of length 512 starting with
"bytearray(iterable_of_ints) -> bytearray
bytearray(string, encoding[, errors]".

In that output, a %SimplePythonObject" is some python object like a str that doesn't reference other python objects and a "%ContainerPythonObject" is a python object that can reference other python objects and so is subject to garbage collection.

Note that the python reference count for every such object is given. This means, in answer to your question about checking for whether your c++ code has done reference counting incorrectly, this will be reflected for you in the change of the reference counts of a given object in the output files. So an error as small as a single reference count error that would lead to a leak can be caught in this way.

%SimplePythonObject can often be detected as leaked, just using "count leaked" with the one core, as long as no allocations (python or native or...) still reference it but the use of comparing the output of "describe used" is more general and offers better coverage for your use case.

Unit test for memory leaks in Python

1 Answers1