0

The following code works fine and prints out an empty list (on python3.4):

import gc
# code interfacing with C and cython
print(gc.garbage, flush=True)

Appending this line to the very end makes it segfault:

gc.collect()

It seems like it is pure luck that the automatic collection does not happen on its own (it took me some time to get reproducible error). The collection is not ran at "end of program" either because I am running this interactively.

Some googling led me to believe that some of the C/Cython code is creating/destroying objects without telling python that those objects are already removed from memory. Then python tries to remove them and crashes.

How can I find what those objects are? I thought that they would be in gc.garbage before the crash inducing gc.collect()?

Or are my assumptions completely incorrect?

Prepending an gc.set_debug(gc.DEBUG_STATS | gc.DEBUG_LEAK) to this code leads to a python exception instead of a segfault.

---> print(gc.garbage, flush=True)
     gc.collect()
...
ReferenceError: weakly-referenced object no longer exists

Edit: changed/simplified after comments from one of the answers

Krastanov
  • 6,479
  • 3
  • 29
  • 42

1 Answers1

1

You're probably misunderstanding both gc.garbage and gc.disable(). gc.disable() turns off the cyclic garbage collector, leaving only reference-counting memory management active. There isn't some separate "garbage detector" that keeps detecting cyclic trash even when the garbage collector is turned off; garbage detection is the garbage collector's job.

Also, gc.garbage is only populated with specific kinds of weird garbage the GC couldn't clear. Most garbage never ends up there.

As for how you'd go about debugging this, I'm not familiar with the tools you'd use. Generic tools like Valgrind would probably be useful, and GC flags like gc.DEBUG_LEAK might help.

user2357112
  • 260,549
  • 28
  • 431
  • 505
  • I disagree. Non-cyclic garbage is collected even when `gc.dissable()` is invoked. What `gc.dissable()` does is to stop the garbage collector from collecting cyclic structures. Am I wrong about that? Also, I am already using `gc.DEBUG_LEAK` - this should ensure that "normal" garbage goes to the `gc.garbage` list. Finally, the problem persist (`gc.garbage` crashes) even if I do not disable the `gc` with `gc.disable()`. – Krastanov Oct 24 '15 at 04:02
  • 1
    @Krastanov: Reference counting is still active, but the cyclic garbage detection and collection is completely disabled. – user2357112 Oct 24 '15 at 04:05