Why do circular imports cause problems with object identity using `isinstance`?

Question

After a few hours of isolating a bug, I have come up with the following MCVE example to demonstrate the problem I've had:

a.py:

from b import get_foo_indirectly

class Foo:
    pass

if __name__ == '__main__':
    print("Indirect:", isinstance(get_foo_indirectly(), Foo))
    print("Direct:", isinstance(Foo(), Foo))

b.py:

def get_foo_indirectly():
    from a import Foo
    return Foo()

The expected output of a.py is:

Indirect: True
Direct: True

The actual output is:

Indirect: False
Direct: True

Moreover, if I create a separate module c.py, the output is as expected:

from a import Foo
from b import get_foo_indirectly

if __name__ == '__main__':
    print("Indirect:", isinstance(get_foo_indirectly(), Foo))
    print("Direct:", isinstance(Foo(), Foo))

Clearly, the interaction between isinstance and the import machinery is not behaving quite like I expected it to. It seems like the use of circular imports has bitten me hard. Why? Is this Python's expected behavior?

Note that this is very oversimplified of the actual context in which I encountered this behavior; modules a and b were both large modules, and b was separated because it had a distinct purpose from a. Now that I've seen the consequences of circular imports, I will probably combine them, perhaps relegating some of the long-winded behavior in b.

@ehacinom Because of the nature of circular imports (though I clearly do not understand it completely); if you import `a` in `b` and `b` in `a` at the same time, you get an error: `ImportError: cannot import name 'get_foo_indirectly' from 'b'` — Graham, Dec 06 '18 at 19:27
If you print off `Foo()` in `a.py` you get `__main__.Foo instance`. If you print off `Foo()` in `b.py` you get `a.Foo instance` — ehacinom, Dec 06 '18 at 19:31
This is a duplicate of [python namespace: __main__.Class not isinstance of package.Class](https://stackoverflow.com/questions/15159854/python-namespace-main-class-not-isinstance-of-package-class) — MEE, Dec 06 '18 at 19:38
@John Good find. However, my example is more succinct, and asks a more general question. I'm not sure if it's worth marking as a duplicate (though you and anyone else can certainly flag and vote to close as a duplicate if you feel differently.) — Graham, Dec 06 '18 at 19:43

score 8 · Accepted Answer · answered Dec 06 '18 at 19:41

8

When you run a Python script it automatically assumes the name __main__. At the time you imported a.py in b.py Python assumed the usual module name (i.e. the name of the file), and at runtime Python changed to __main__ because it's the entry point script; so, it's like the Foo class was declared in two different places: the __main__ module and the a module.

You're then comparing an instance of a.Foo (created inside get_foo_indirectly) and __main__.Foo.

This was already discussed here.

If you need to do circular imports, don't put the entry point script in the loop. This way you avoid this --very consusing-- behaviour of Python.

answered Dec 06 '18 at 19:41

Gabriel

1,922
2
19
37

By don't put the "entry point script in the loop", do you mean use a separate module like my `c.py` example? – Graham Dec 06 '18 at 19:49
1

@Graham Yes! The entry point module is the module you call in the terminal, the one with `if __name__ == '__main__'`. – Gabriel Dec 06 '18 at 19:50

Rocky Li · Answer 2 · 2018-12-06T19:56:11.223

I've ran the same script, what I found is that you can add some line to expose some interesting differences:

from b import get_foo_indirectly

class Foo:
    pass

if __name__ == '__main__':
    print("Indirect:", isinstance(get_foo_indirectly(), Foo))
    print(type(get_foo_indirectly()))
    print("Direct:", isinstance(Foo(), Foo))
    print(type(Foo()))

Output:

Indirect: False
<class 'a.Foo'>
Direct: True
<class '__main__.Foo'>

Now, for your c.py example, both are a.Foo, so they evaluate to the same. What it seem to infer here is that objects are also traced to the file/module path they come from.

This is an important distinction that act beyond the calling of __main__, (Which assign path as __main__, instead of current path from PATH, @Gabriel, @ehacinom). Assuming you have the exact same class defined in different files, for example, d.py:

class Foo:
    pass

And you try to import them to the same class e.py:

from a import Foo
from d import Foo as Fooo

print(type(Foo()))
print(type(Fooo()))

You'll get:

<class 'a.Foo'>
<class 'd.Foo'>

This is how python namespace classes. Moreover, if you move d.py to directory /d with __init__.py within the directory, and the class would become

<class 'd.d.Foo'>

All path are relative to the python PATH. The modules installed in site_packages would be available on PATH, and type would return the path of objects starting from base directory, etc:

<class 'matplotlib.figure.Figure'>

ehacinom · Answer 3 · 2018-12-06T19:41:23.890

Doing an import within __main__ must change where the class is namespaced.

__name__ is a built-in variable which evaluates to the name of the current module. You're overwritting the special variable __name__ == __main__ at the end of the a.py file, and importing Foo within that context.

If you print off Foo() in a.py __main__() you get __main__.Foo instance

If you print off Foo() in b.py get_foo_indirectly() you get a.Foo instance

Because of the circular import, in b.py you import Foo within the function, which is called in __main__. A similar thing happens if you define a class in a terminal -- it gets namespaced to __console__.

Why do circular imports cause problems with object identity using `isinstance`?

3 Answers3