Python `__init__.py` and initialization of objects in a code

Question

I've read the documentation about __init__.py files and some nice questions here on SO, but I'm still confused about its proper usage.

Context

I have a code with many packages and sub-packages. I've defined many classes, some of which I need to create one (and only one) instance for the whole user session. These new objects are then used in different parts of code, so that anytime I (or the user) update data/information in this objects, it will be used across all the code without having to change anything else. To be clearer, let me show you a basic scheme of what I'm talking about.

The code have an over-simplified structure like:

root/
    __init__.py
    tools/
        __init__.py
        ... (some modules)
        aliases.py (which defines the class Aliases)
    physics/
        __init__.py
        ... (some modules)
        constants/
            ... (some modules)
            constants.py (which defines the class Ctes)
        units.py (which defines the class Units)

In the code, I need to manage aliases, units and constants. The way I found to deal with that is to create one instance of each, and use it across all the code. With this method, I'm sure, for example, that if an alias is added while the code is still running, it could be used anywhere in the code because there is only one shared instance of Aliases. This is what I need (the same apply for units, and constants by the way).

Current status

As for now, the way I'm doing this is, I think, not the best. Indeed, I'm creating the instance of, let's say Aliases, directly after declaring the class, in the same file:

in root/tools/aliases.py

import ... (stuff

class Aliases(object):
    """The Aliases manager"""
    ...

ALIASES = Aliases()

And then, in any file I need to use Aliase, I do:

in any_file.py (anywhere in the code)

from root.tools.aliases import ALIASES

ALIASES.method1() # possibly in functions or other classes
ALIASES.method2() # ... etc

And for some other classes, I'm even using the __init__.py file at the root of the code:

in root/__init__.py

# CTES is the instance of Ctes() created in root/physics/constants/constants.py
from root.physics.constants.constants import CTES
CTES.add(...) # add a new constant that needs to be known

(of course, CTES does not just store some constants, I define some methods to exploit them, so it makes sense to have them in this class instead of just defining them as regular python constants in a module)

Questions

I'm wondering if I'm doing this right (probably not). Maybe it is better to use files __init__.py and initiate the shared instances in it. But then do this brings some problems (like dependency cycles, or increased memory usage...)? Also, how to use the created instances elsewhere in the code? Like this?:

in root/tools/__init__.py

import root.tools.aliases as Al
ALIASES = Al.Aliases()
# should I delete the imported module: del Al ??

and then in any_file.py

from root.tools import ALIASES
ALIASES.method(...)

Or should all these instances better be included in a file (eg. root/shared.py) which I import in root/__init__.py so that I'm sure it is initiated?

I've read many times it is better to keep __init__.py files empty (which is the case right now in my code, except of course for root/__init__.py). What do you think?

I'm a bit lost (you could probably see that from the fact I'm not very clear). Any help/advice is more than welcome. I'd like to avoid any non pythonic solution, or solutions that could confuse the user, or make things unsafe.

Answer 1

Use the Borg pattern to implement all of your shared classes throughout the code and then store them in their own modules (or bundled together) wherever that may seem appropriate.

Then simply import them where needed and instantiate the class. They will all share the same context and you can even fine tune the behavior of the instances per instantiation.

If they are isolated in their own module(s) then you won't have any cyclic dependencies and you will minimize parsing time for the interpreter during imports. Best yet you won't pollute your namespace by importing global variables elsewhere. As your function goes out of scope the instance will be tagged for deletion, but the state will be preserved with the class definition.

Answer 2

The thing you are doing where you create a single instance inline in your module is FINE. There's nothing wrong with using global variables (except when they're not appropriate, but that applies to any language feature). The only potential disadvantage is if you will import that module without using it and waste resources initializing that class. Unless your class is doing something really heavy at creation time, which I very much doubt, that effect is negligible. The only thing I would suggest is that you adjust your naming. Your class should probably begin with an underscore so users of your module know not to touch it, and your instance should be lowercase. The method you are using will not cause any bad side-effects.

There's no one saying you HAVE to put all your classes in their own file, or that you HAVE to use classes at all. Maybe an alias module with functions in it and "private" state in module globals makes more sense for you. Since you're not using the object system to get multiple instances with independent state, the only reason to make a class is if that is how you prefer to organize your code and your API. The most important thing here is that you're happy with how your code is organized and with how your modules will be used. You're much more likely to confuse users by using object-oriented techniques too much than too little imo. Just make sure you make a class when you really do need one.

There are plenty of other approaches to singletons. This is the first I've heard of the "Borg pattern", and it sounds wasteful and silly to me, but I guess it would at least technically work, and the waste at least is negligible. You can write a function that instantiates a class the first time it's called, stores that in a global, and returns the global in subsequent calls (but then you have to worry about threading, which is not a problem the way you're already doing it). You can make a class whose __init__ method raises TypeError, so that it cannot be instantiated at all, and do everything using classmethods or staticmethods, keeping all the state you need on the class itself. It's even possible to make a class that returns the same instance every time. The result of all of these is that you get an object out of your module that you then use in your code, and the way you use it after that is going to look the same.

I think you should just instantiate your class inline in one of your modules, as in the code you posted, because it works, it is a common pattern, it's sensible, and it makes sense to you.

There is no reason to ever delete a module that you've imported. It doesn't save you anything. The module exists forever once imported (unless you actively do some weird things to get rid of it), and globals aren't some precious resource that you need to conserve.

You will not introduce a circular dependency by importing your modules inside __init__.py, as long as you refer to your modules internally with a relative name (i.e. aliases instead of root.tools.aliases). Circular dependencies are a real problem, so be careful about that. As long as your modules are only importing things that are deeper in the tree, and that you think of as "lower-level" (meaning those lower-level modules won't import the higher-level things that might be using them), you should be OK. I would caution against putting any substantial code in __init__.py, but I think instantiating your classes is fine if that's where you want the instances to live.