1

While writing a Python package I encountered some strange behavior. This is my directory layout:

my_package/
    __init.py__
    client.py
    events.py
    utils.py

In __init__.py there are some from ... imports that import various names:

__all__ = ['Client', 'get_websocket_url', 'format_cookies']

from .client import ChatSession as Client
from .utils import get_websocket_url, format_cookies

When I import my_package and run dir(my_package), it seems that client.py, events.py, and utils.py are all included as attributes:

['Client', '__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', 'client', 'events', 'format_cookies', 'get_websocket_url', 'utils']

What is the reason that this happens? I was able to remove those attributes by deleting them in __init__.py but that doesn't look very good to linters. The only other file that imports anything is client.py:

from .events import Event
from .utils import get_websocket_url

...

But this doesn't import any module as a whole, so this is puzzling me. Any thoughts?

miike3459
  • 1,431
  • 2
  • 16
  • 32
  • 2
    After a module is imported it becomes an attribute of its package. That's the intended and specified behavior. What is the actual issue you have with this? – Klaus D. Sep 25 '19 at 23:25
  • @KlausD. Well, I want to be able to stop the three src files from being added as attributes to the module. It seems like most packages don't exhibit behavior like this - is there something that I'm doing differently of incorrect? Also forgive me because I'm pretty new to the packaging system. – miike3459 Sep 25 '19 at 23:27
  • 1
    Well, if you don't want the modules to be in the package, then you don't put the source files there. Why would you want to have modules in a packages without being in the package? – Klaus D. Sep 25 '19 at 23:35
  • @KlausD. I can import the names I want in `__init__.py`, so I don't see any point for those module attributes to be there. – miike3459 Sep 25 '19 at 23:37
  • I believe this is a consequence of how the relative imports work, I am away from a computer, but try specifying the full import, i.e. `from my_package.client import ChatSession as Client` – juanpa.arrivillaga Sep 25 '19 at 23:47
  • @juanpa.arrivillaga Tried that, unfortunately nothing changed. – miike3459 Sep 25 '19 at 23:50
  • 2
    So, how would you import them with them without being an attribute? The import syntax uses them as one, even that this is a bit harder to see in relative import and that if not already imported the attribute will be created while importing. – Klaus D. Sep 26 '19 at 00:07
  • @KlausD. since we've been discussing this with the OP for a while in chat: they essentially want to do (the relative version of) `from foo import bar` without `foo` appearing in the current namespace. The surprising bit they're asking about is why `foo` gets imported on its own when `from foo import bar` would only suggest to put the name `bar` into the namespace, not `foo`. Does this help clarify? – Andras Deak -- Слава Україні Sep 26 '19 at 00:10
  • 1
    Because that is how Python works, how it is specified and documented. And that they (?) have still not explained: what problem does this cause? Stack Overflow is not the right place to just complain about Python language design decisions. – Klaus D. Sep 26 '19 at 00:19
  • @KlausD. I don't think they are complaining, the question seems to be "why does this happen?". If it's specified and documented it's all well, but OP certainly hasn't found where it's documented and no links to this behaviour have been posted yet. A short explanation pointing to the relevant part of the docs would be an answer to this question. – Andras Deak -- Слава Україні Sep 26 '19 at 00:21
  • @KlausD. Also, to add - considering I don't see this behavior in most common Python packages, is there a better package layout that you think would work better for the situation? – miike3459 Sep 26 '19 at 00:22

1 Answers1

1

Why does __init__.py import all files in the folder as modules?

It does not. You must explicitly import them. Add an empty other.py file in the package directory and you will not see it does not get imported at all, unless you also add the code to trigger the submodule import. See Do I need to import submodules directly? for more about that.

You've probably meant to ask something more like "Why do submodule imports always bind names in the parent module?" as a question title. Well, this is just how the import system in Python works. It's documented under 5.4.2. Submodules:

When a submodule is loaded using any mechanism (e.g. importlib APIs, the import or import-from statements, or built-in __import__()) a binding is placed in the parent module’s namespace to the submodule object. For example, if package spam has a submodule foo, after importing spam.foo, spam will have an attribute foo which is bound to the submodule.

To give you a better intuition about why this is actually a sensible behavior, consider that importing the parent module (my_module) creates an item in sys.modules dict, with the key being "my_module" (string) and the value being my_module (module). Subsequently, the submodule import of client will also create an item in sys.modules with the key being "my_module.client" (string) and the value being my_module.client (module). Since both my_module and my_module.client are now cached in the sys.modules, it makes sense for client to be attached as an attribute on my_module.

If you don't like the submodule name bindings appearing for whatever reason, you may simply remove those submodule names from the parent module namespace by placing a del statement after the import-from statements.

wim
  • 338,267
  • 99
  • 616
  • 750