Should I worry about circular references in Python?

Question

Suppose I have code that maintains a parent/children structure. In such a structure I get circular references, where a child points to a parent and a parent points to a child. Should I worry about them? I'm using Python 2.5.

I am concerned that they will not be garbage collected and the application will eventually consume all memory.

score 40 · Accepted Answer · answered Mar 11 '10 at 21:37

"Worry" is misplaced, but if your program turns out to be slow, consume more memory than expected, or have strange inexplicable pauses, the cause is indeed likely to be in those garbage reference loops -- they need to be garbage collected by a different procedure than "normal" (acyclic) reference graphs, and that collection is occasional and may be slow if you have a lot of objects tied up in such loops (the cyclical-garbage collection is also inhibited if an object in the loop has a __del__ special method).

So, reference loops will not affect your program's correctness, but may affect its performance and/or footprint.

If and when you want to remove unwanted loops of references, you can often use the weakref module in Python's standard library.

If and when you want to exert more direct control (or perform debugging, see what exactly is happening) regarding cyclical garbage collection, use the gc module in Python's standard library.

Plus 1 for the note about `__del__`. If your object destructors have side effects, then you may want to think about cyclic references (and when things get destroyed) a bit more carefully. — speedplane, Oct 04 '17 at 00:35

score 18 · Answer 2 · edited Jun 16 '12 at 12:24

18

Experimentally: you're fine:

import itertools

for i in itertools.count():
    a = {}
    b = {"a":a}
    a["b"] = b

It consistently stays at using 3.6 MB of RAM.

edited Jun 16 '12 at 12:24

Peter Mortensen

30,738
21
105
131

answered Mar 11 '10 at 20:18

cobbal

69,903
20
143
156

Which implementation did you use? – Display Name Jun 12 '14 at 12:56
@SargeBorsch CPython 2.something. I imagine any of the major implementations would behave the same way though. – cobbal Jun 12 '14 at 17:02
That's short term object life cycle and no more than 2 are used at any time. Can we expect the same in other scenarios? – Jimmy T. Jul 05 '21 at 18:01
@JimmyT. TL;DR: No, nothing is guaranteed, and you can't count on specific garbage collection strategies of any python implementation. It probably depends on the reference-counted implementation of the python interpreter. So probably you can count on the behavior in any situation where CPython is used, and no closures or other references are grabbing onto the values, but you probably can't count on it where a pure mark-and-sweep or stop-and-copy implementation is used. In short, nothing is guaranteed. It all depends. Many different garbage collectors are valid by the standard. – cobbal Jul 06 '21 at 05:59

score 13 · Answer 3 · edited Mar 11 '10 at 20:26

13

Python will detect the cycle and release the memory when there are no outside references.

edited Mar 11 '10 at 20:26

S.Lott

384,516
81
508
779

answered Mar 11 '10 at 20:17

Ned Batchelder

364,293
75
561
662

2

Assuming of course there are no `__del__` methods. Which there normally shouldn't be, but you never know. For a while, even `collections.OrderedDict` had one for some reason. – Antimony Oct 01 '12 at 04:52

score 7 · Answer 4 · answered Mar 11 '10 at 20:18

7

Circular references are a normal thing to do, so I don't see a reason to be worried about them. Many tree algorithms require that each node have links to its children and its parent. They're also required to implement something like a doubly linked list.

answered Mar 11 '10 at 20:18

Colin

10,447
11
46
54

1

Thanks Colin. I didn't know they were "a normal thing." They seemed very special to me. But now I learned otherwise. :) – bodacydo Mar 11 '10 at 20:21
Also, they are obviously required for graphs. – Antimony Oct 01 '12 at 04:41

Finn Årup Nielsen · Answer 5 · 2012-11-22T11:41:12.133

There seems to be a issue with references to methods in lists in a variable. Here are two examples. The first one does not call __del__. The second one with weakref is ok for __del__. However, in this later case the problem is that you cannot weakly reference methods: http://docs.python.org/2/library/weakref.html

import sys, weakref

class One():
    def __init__(self):
        self.counters = [ self.count ]
    def __del__(self):
        print("__del__ called")
    def count(self):
        print(sys.getrefcount(self))


sys.getrefcount(One)
one = One()
sys.getrefcount(One)
del one
sys.getrefcount(One)


class Two():
    def __init__(self):
        self.counters = [ weakref.ref(self.count) ]
    def __del__(self):
        print("__del__ called")
    def count(self):
        print(sys.getrefcount(self))


sys.getrefcount(Two)
two = Two()
sys.getrefcount(Two)
del two
sys.getrefcount(Two)

score 3 · Answer 6 · answered Mar 11 '10 at 20:20

3

I don't think you should worry. Try the following program and will you see that it won't consume all memory:

while True:
    a=range(100)
    b=range(100)
    a.append(b)
    b.append(a)
    a.append(a)
    b.append(b)

answered Mar 11 '10 at 20:20

douglaz

1,306
2
13
17

don't you mean `a.extend(b)`, not `append`? – richizy Feb 23 '14 at 19:47
4

@richizy I really mean append because I want to save the reference to a and b inside a and b, not the values. In this way the circular reference will happen. – douglaz Feb 25 '14 at 02:55

Should I worry about circular references in Python?

6 Answers6

Linked