Python CApi Reference Count Details

Question

I'm looking at some example code here ( https://docs.python.org/2.0/api/refcountDetails.html ) and trying to get a better understanding of the difference between two of the examples: The first example is:

PyObject *t;

t = PyTuple_New(3);
PyTuple_SetItem(t, 0, PyInt_FromLong(1L));
PyTuple_SetItem(t, 1, PyInt_FromLong(2L));
PyTuple_SetItem(t, 2, PyString_FromString("three"));

The author explains that PyTuple_SetItem() steals the reference (so there is no need to DECREF it). Fine, I get that. The author then presents similar code using PySequence_SetItem() which does not steal the reference, so the caller must DECREF, and the example code looks like this:

PyObject *l, *x;

l = PyList_New(3);
x = PyInt_FromLong(1L);
PySequence_SetItem(l, 0, x); Py_DECREF(x);
x = PyInt_FromLong(2L);
PySequence_SetItem(l, 1, x); Py_DECREF(x);
x = PyString_FromString("three");
PySequence_SetItem(l, 2, x); Py_DECREF(x);
PyObject *l, *x;

My question is what would happen if the 2nd example were similar to the first in passing PyTYPE_FromSOMETYPE as follows?

PyObject *l;

l = PyList_New(3);
PySequence_SetItem(l, 0, PyInt_FromLong(1L));
PySequence_SetItem(l, 1, PyInt_FromLong(2L));
PySequence_SetItem(l, 2, PyString_FromString("three"));

Is this last case benign, or does it cause a memory leak (because PySequence_SetItem will not take ownership of the reference created by PyInt_FromLong and PyString_FromString, and neither does the caller DECREF it)??

Answer 1

It causes a memory leak.

When you create an object, it starts off with a refcount of 1. The object only gets deleted if the refcount goes to 0.

First example: When you pass your new object to a function that steals a reference (takes ownership), like PyTuple_SetItem, the refcount is not incremented, so it's still 1. When the tuple eventually gets destroyed and decrefs all of its elements, the count will drop to 0, so it'll be destroyed. All is good.

Third example: When you pass your new object to a function that does not steal a reference (makes a new reference), like PySequence_SetItem, the refcount is incremented, so it's 2. When the tuple eventually gets destroyed and decrefs all of its elements, the count will drop to 1, so it won't be destroyed. And, since nobody else has a reference to it anymore (unless you stored it somewhere), there's no way anyone can ever decref it. So it's leaked.

Second example: When you pass your new object to a function that does not steal a reference (makes a new reference), like PySequence_SetItem, but then call Py_DECREF on it, the refcount is incremented to 2 and decremented back to 1. So, when the tuple eventually gets destroyed and decrefs all of its elements, the count will drop to 0. All is good again.

---

If you're wondering why Python would both with any non-stealing functions, you just need to think of a less trivial case.

What if you wanted to put the item in two tuples instead of one? Or if you wanted to put it in a tuple, but also store it in a C static pointer, or in some module's globals, or somewhere else? You want the reference count to be bumped up by 2 if you want to store it in two places, and them dropped back down by 1 when your local variable goes away. For the really simple case where you're just creating something and immediately handing it off, reference-stealing functions let you avoid that one-incref-and-one-decref, as well as being nice and convenient for one-lines. But for anything more complicated, that doesn't make sense.