In Python, is it possible to expose modules from subpackages at package level?

Question

I have the following conundrum. I'm trying to expose some modules from a subpackage of a package at the parent package level.

The folder structure is the essentially like this:

script.py
package/
    __init__.py
    module1.py
    subpackage/
        __init__.py
        submodule1.py
        submodule2.py

In the script.py file I currently have to write

from package.subpackage.submodule1 import foo

if I want to import something from the submodule1.py file, but I would like to be able to expose the files submodule1.py and submodule2.py at package level, so that all my imports can look like

from package.module1 import bar
from package.submodule1 import foo
from package.submodule2 import goo

Note that I don't want to expose bar, foo and goo at package level, i.e. not

from package import bar
from package import foo

because the separation between modules is still important in my case.

Is this even possible? Is there a trick in the __init__.py file to do so?

Thanks!

MisterMiyagi · Answer 1 · 2016-06-30T19:43:09.887

11

For your purpose, python modules are just namespaces. That is, everything in the globals of a module can be imported and used as module.thingy. You may have noticed that in many modules, you also find some builtins. For example, logging.os is just the regular os module.

So, in your package (package/__init__.py) import whatever you wish, and bind it to the name you want to expose it as.

# package/__init__.py
# import package.module1 as module1  # one *can* do this, but its at best without benefit (see comments)
import package.subpackage.submodule1 as submodule1
import package.subpackage.submodule2 as submodule2

This allows to do import package.submodule1 and import package.submodule1 as foo.

Note that this simple way will not allow you to do from package.submodule1 import bar. For this, you need an actual dummy module.

# package/submodule1/__init__.py
from package.subpackage.submodule1 import *

edited Jun 30 '16 at 19:43

answered Jun 30 '16 at 18:44

MisterMiyagi

44,374
10
104
119

Just make sure never to shadow subpackage or submodule names. Something like `from .foo import foo` would break imports of the `foo` submodule. – user2357112 Jun 30 '16 at 18:51
@user2357112 Have you tested this? I only tried `os.path = None; from os.path import dirname` and it works fine. The dummy module I mentioned is needed precisely because the import machinery will prefer actual modules (having `__init__.py`) over the parent namespace, it seems. – MisterMiyagi Jun 30 '16 at 18:55
I'm afraid I tried what you suggested, but when running `script.py` I get either a `SyntaxError` if I write as you suggest `import .subpackage.submodule1 as submodule1` or an `ImportError: No module named subpackage.submodule1` if I write `import subpackage.submodule1 as submodule1` (i.e. without prepending the `.`) – Marco Selvi Jun 30 '16 at 19:00
@MarcoSelvi The '.' marks a relative import. It can only be used when inside a package, not in an interactive session. You can replace it with `package.subpackage.submodule2`. – MisterMiyagi Jun 30 '16 at 19:05
1

@MisterMiyagi: But if you try `from os import path`, you get `None`. It becomes really awkward to get at the actual module you shadowed. – user2357112 Jun 30 '16 at 19:38
@user2357112 Ah, you are right. Ugly indeed, but should work if the module is assigned to its own name. Granted, there's no benefit from it either. I'll remove that line from the example. Thanks for testing. – MisterMiyagi Jun 30 '16 at 19:42

score 6 · Answer 2 · answered Jul 01 '16 at 19:27

Yes It's possible.
Let's see what happens step by step;

When you do from package.submodule1 import foo, sys.modules is checked if it has package.
If not found package/__init__.py is run and loaded
Again sys.modules is checked for package.submodule1
If not found, package/submodule1.py is checked for existence.

We can make step 3 pass by making an entry in sys.modules for package.submodule1 in step 2.

package/__init__.py

import sys
from .subpackage import submodule1
from .subpackage import submodule2

for module in (submodule1, submodule2):
    full_name = '{}.{}'.format(__package__, module.__name__.rsplit('.')[-1])
    sys.modules[full_name] = sys.modules[module.__name__]

In Python, is it possible to expose modules from subpackages at package level?

2 Answers2