3

I have a python application that has some performance hiccups. I want to add the garbage collector's events (especially, when does it get called) to my logs. Is it possible?

thanks.

r0u1i
  • 3,526
  • 6
  • 28
  • 36

2 Answers2

6

http://docs.python.org/library/gc.html#gc.set_debug

You can set the Flags, but they get written to stderr.

Available flags are

gc.DEBUG_STATS
gc.DEBUG_COLLECTABLE
gc.DEBUG_UNCOLLECTABLE
gc.DEBUG_INSTANCES
gc.DEBUG_OBJECTS
gc.DEBUG_SAVEALL
gc.DEBUG_LEAK

ALSO

When you are dealing with performance, you might wish to profile your code for exhaustive Loops or Function calls. You can use cProfile or hotshot. More here http://docs.python.org/library/profile.html

0 _
  • 10,524
  • 11
  • 77
  • 109
meson10
  • 1,934
  • 14
  • 21
  • thanks for the performance tip, but my application has those hiccups in sporadic places, that usually don't involve loops or recursion. That's why I suspect the GC – r0u1i Nov 18 '11 at 08:09
1

Python (at least CPython version 2.x) uses reference counting to implement its garbage collector (cf. Why Java and Python garbage collection methods are different?), so it's not really "called", as it is in Java.

Refcounting means that each time a new reference to a given object is created, a counter is incremented, each time the reference is lost (end of scope, reassignment, ...), the counter is decremented. When it reaches 0, the object memory is freed.

So the solution Python offers to your problem is to override the __del__ method of the object:

class test:
    def __del__(self):
         #self is about to be freed, do what ever you want 
         pass

EDIT: According to the link above, there is another mechanism running periodically:

CPython (reference counting is not part of python itself, but part of the C implementation thereof) catches circular references with a separate garbage collection routine that it runs periodically...

but only involved in case of circular references.

EDIT 2: As mentioned in the comments and here, __del__ is not the safest solution. Here is a certainly better way to achieve a similar behavior:

import weakref

class test:
    pass

t = test()

def prepare_cb(obj):
  #save information about the obj
  uid = id(obj)

  def do_some_logging(weak):
     print "Object %s cleaned up" % uid

  return do_some_logging

weak = weakref.ref(t, prepare_cb(t))

del t
Community
  • 1
  • 1
Kevin
  • 4,618
  • 3
  • 38
  • 61
  • This will only work for the custom classes and not for the built-ins. Also, "ALL" the classes in the source code will have to be edited. – meson10 Nov 18 '11 at 08:12
  • 1
    The GC module only deals with cyclic references. Defining `__del__` is a actually a terrible idea because it makes the object uncollectable by the cyclic GC. – John La Rooy Nov 18 '11 at 08:15
  • @gnibbler: good point, the [doc](http://docs.python.org/library/gc.html#gc.garbage) is clear about that. So it depends how frequent are the cyclic refs in the code, and what you actually want to know about garbage collection. – Kevin Nov 18 '11 at 09:01