5

So if I have a list a and append a to it, I will get a list that contains it own reference.

>>> a = [1,2]
>>> a.append(a)
>>> a
[1, 2, [...]]
>>> a[-1][-1][-1]
[1, 2, [...]]

And this basically results in seemingly infinite recursions.

And not only in lists, dictionaries as well:

>>> b = {'a':1,'b':2}
>>> b['c'] = b
>>> b
{'a': 1, 'b': 2, 'c': {...}}

It could have been a good way to store the list in last element and modify other elements, but that wouldn't work as the change will be seen in every recursive reference.

I get why this happens, i.e. due to their mutability. However, I am interested in actual use-cases of this behavior. Can somebody enlighten me?

Sayandip Dutta
  • 15,602
  • 4
  • 23
  • 52
  • Are you specifically asking about why circular references are useful or why changes to one instance of the reference propagate to others? – Xetera Nov 28 '19 at 07:37
  • I am specifically asking about it's usefulness. A scenario where this behavior would be useful. – Sayandip Dutta Nov 28 '19 at 07:38

5 Answers5

12

The use case is that Python is a dynamically typed language, where anything can reference anything, including itself.

List elements are references to other objects, just like variable names and attributes and the keys and values in dictionaries. The references are not typed, variables or lists are not restricted to only referencing, say, integers or floating point values. Every reference can reference any valid Python object. (Python is also strongly typed, in that the objects have a specific type that won't just change; strings remain strings, lists stay lists).

So, because Python is dynamically typed, the following:

foo = []
# ...
foo = False

is valid, because foo isn't restricted to a specific type of object, and the same goes for Python list objects.

The moment your language allows this, you have to account for recursive structures, because containers are allowed to reference themselves, directly or indirectly. The list representation takes this into account by not blowing up when you do this and ask for a string representation. It is instead showing you a [...] entry when there is a circular reference. This happens not just for direct references either, you can create an indirect reference too:

>>> foo = []
>>> bar = []
>>> foo.append(bar)
>>> bar.append(foo)
>>> foo
[[[...]]]

foo is the outermost [/]] pair and the [...] entry. bar is the [/] pair in the middle.

There are plenty of practical situations where you'd want a self-referencing (circular) structure. The built-in OrderedDict object uses a circular linked list to track item order, for example. This is not normally easily visible as there is a C-optimised version of the type, but we can force the Python interpreter to use the pure-Python version (you want to use a fresh interpreter, this is kind-of hackish):

>>> import sys
>>> class ImportFailedModule:
...     def __getattr__(self, name):
...         raise ImportError
...
>>> sys.modules["_collections"] = ImportFailedModule()  # block the extension module from being loaded
>>> del sys.modules["collections"]  # force a re-import
>>> from collections import OrderedDict

now we have a pure-python version we can introspect:

>>> od = OrderedDict()
>>> vars(od)
{'_OrderedDict__hardroot': <collections._Link object at 0x10a854e00>, '_OrderedDict__root': <weakproxy at 0x10a861130 to _Link at 0x10a854e00>, '_OrderedDict__map': {}}

Because this ordered dict is empty, the root references itself:

>>> od._OrderedDict__root.next is od._OrderedDict__root
True

just like a list can reference itself. Add a key or two and the linked list grows, but remains linked to itself, eventually:

>>> od["foo"] = "bar"
>>> od._OrderedDict__root.next is od._OrderedDict__root
False
>>> od._OrderedDict__root.next.next is od._OrderedDict__root
True
>>> od["spam"] = 42
>>> od._OrderedDict__root.next.next is od._OrderedDict__root
False
>>> od._OrderedDict__root.next.next.next is od._OrderedDict__root
True

The circular linked list makes it easy to alter the key ordering without having to rebuild the whole underlying hash table.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • Sure but this is mostly circular linked objects. So this raise a question do i have to do anything special to prevent infinity recursion when i implement __str__ on object – Lee Dec 04 '19 at 13:55
  • @Lee: you can use the [`reprlib.recursive_repr()` decorator](https://docs.python.org/3/library/reprlib.html#reprlib.recursive_repr) to provide a fill value when the same object is encountered during recursing. It handles handling remembering if an object has been seen already, in a thread-safe manner. – Martijn Pieters Dec 04 '19 at 18:19
  • May I ask why the linked list has to be curcular to "alter the key ordering without having to rebuild the whole underlying hash table"? – laike9m Dec 10 '19 at 07:11
  • @laike9m: it doesn't have to be a circular linked list, but the implementation is much simpler because it is (no need to special case handling an empty dict, for example). – Martijn Pieters Dec 12 '19 at 18:25
5

However, I am interested in actual use-cases of this behavior. Can somebody enlighten me?

I don't think there are many useful use-cases for this. The reason this is allowed is because there could be some actual use-cases for it and forbidding it would make the performance of these containers worse or increase their memory usage.

Python is dynamically typed and you can add any Python object to a list. That means one would need to make special precautions to forbid adding a list to itself. This is different from (most) typed-languages where this cannot happen because of the typing-system.

So in order to forbid such recursive data-structures one would either need to check on every addition/insertion/mutation if the newly added object already participates in a higher layer of the data-structure. That means in the worst case it has to check if the newly added element is anywhere where it could participate in a recursive data-structure. The problem here is that the same list can be referenced in multiple places and can be part of multiple data-structures already and data-structures such as list/dict can be (almost) arbitrarily deep. That detection would be either slow (e.g. linear search) or would take quite a bit of memory (lookup). So it's cheaper to simply allow it.

The reason why Python detects this when printing is that you don't want the interpreter entering an infinite loop, or get a RecursionError, or StackOverflow. That's why for some operations like printing (but also deepcopy) Python temporarily creates a lookup to detect these recursive data-structures and handles them appropriately.

MSeifert
  • 145,886
  • 38
  • 333
  • 352
3

Consider building a state machine that parse string of digits an check if you can divide by 25 you could model each node as list with 10 outgoing directions consider some connections going to them self

def canDiv25(s):
  n0,n1,n1g,n2=[],[],[],[]
  n0.extend((n1,n0,n2,n0,n0,n1,n0,n2,n0,n0))
  n1.extend((n1g,n0,n2,n0,n0,n1,n0,n2,n0,n0))
  n1g.extend(n1)
  n2.extend((n1,n0,n2,n0,n0,n1g,n0,n2,n0,n0))
  cn=n0
  for c in s:
    cn=cn[int(c)]
  return cn is n1g

for i in range(144):
  print("%d %d"%(i,canDiv25(str(i))),end='\t')

While this state machine by itself has little practical it show what could happen. Alternative you could have an simple Adventure game where each room is represented as a dictionary you can go for example NORTH but in that room there is of course a back link to SOUTH. Also sometimes game developers make it so that for example to simulate a tricky path in some dungeon the way in NORTH direction will point to the room itself.

Lee
  • 1,427
  • 9
  • 17
3

A very simple application of this would be a circular linked list where the last node in a list references the first node. These are useful for creating infinite resources, state machines or graphs in general.

def to_circular_list(items):
  head, *tail = items
  first = { "elem": head }
  current = first
  for item in tail:
    current['next'] = { "elem": item }
    current = current['next']
  current['next'] = first
  return first

to_circular_list([1, 2, 3, 4])

If it's not obvious how that relates to having a self-referencing object, think about what would happen if you only called to_circular_list([1]), you would end up with a data structure that looks like

item = {
  "elem": 1,
  "next": item
}

If the language didn't support this kind of direct self referencing, it would be impossible to use circular linked lists and many other concepts that rely on self references as a tool in Python.

Xetera
  • 1,390
  • 1
  • 10
  • 17
3

The reason this is possible is simply because the syntax of Python doesn't prohibit it, much in the way any C or C++ object can contain a reference to itself. An example might be: https://www.geeksforgeeks.org/self-referential-structures/

As @MSeifert said, you will generally get a RecursionError at some point if you're trying to access the list repeatedly from itself. Code that uses this pattern like this:

a = [1, 2]
a.append(a)

def loop(l):
    for item in l:
        if isinstance(item, list):
            loop(l)
        else: print(item)

will eventually crash without some sort of condition. I believe that even print(a) will also crash. However:

a = [1, 2]
while True:
    for item in a:
        print(item)

will run infinitely with the same expected output as the above. Very few recursive problems don't unravel into a simple while loop. For an example of recursive problems that do require a self-referential structure, look up Ackermann's function: http://mathworld.wolfram.com/AckermannFunction.html. This function could be modified to use a self-referential list.

There is certainly precedent for self-referential containers or tree structures, particularly in math, but on a computer they are all limited by the size of the call stack and CPU time, making it impractical to investigate them without some sort of constraint.

Alex Weavers
  • 710
  • 3
  • 9