2

Say we have this file structure:

project/
- ticklers/
  - kitten_tickler.py
    - class KittenTickler
  - puppy_tickler.py
    - class PuppyTickler

Assume KittenTickler has enough complexity to not want to share its file with PuppyTickler. The fully qualified names of the defined classes become project.ticklers.kitten_tickler.KittenTickler and project.ticklers.puppy_tickler.PuppyTickler. This is, obviously, redundant. It works well with the usual Python policy of stuffing multiple classes into one file, but it doesn't fit my project. Is there any way to "skip" the file name in the module chain, aggregating puppy_tickler and kitten_tickler contents into ticklers module? Ultimately, I'm interested in having just project.ticklers.PuppyTickler and project.ticklers.KittenTickler.

I tried putting this into project/ticklers/__init__.py:

from .kitten_tickler import KittenTickler
from .puppy_tickler import PuppyTickler

which results in them being accessible in both packages, but KittenTickler.__module__ still ends up being project.ticklers.kitten_tickler (whereas I'd want it as project.ticklers). Additionally, if I do

from project.ticklers import KittenTickler

I will also inadvertently import PuppyTickler as well. So I am not too happy with this approach. Finally, it is not very maintainable.

Am I fighting a losing battle here, or is there a way for me to declare these classes to my satisfaction?

Amadan
  • 191,408
  • 23
  • 240
  • 301
  • I'm curious: why do the qualified names matter? – AKX Aug 07 '18 at 08:42
  • *"it doesn't fit my project"* - why and/or how not? Could you give some context to the problem you're trying to solve? – jonrsharpe Aug 07 '18 at 08:42
  • 1
    Why does the `__module__` matter? And why is it a problem that the `puppy_tickler` module is imported into memory? – Martijn Pieters Aug 07 '18 at 08:46
  • @AKX: I'm dynamically loading components from a configuration file, and repetitive names look ugly, increase noise and make mistakes more likely. It's mostly an aesthetics thing. The world will not end if I don't solve this, but I've been thinking about this for a while and I have a hard time accepting it might not be possible (or it'll be just another thing I hate about Python). – Amadan Aug 07 '18 at 08:46
  • You can't have it both ways, either you have a flatter namespace from which to import (and Python loads both modules for you in the process), or you have a bushy namespace and get to import individual modules. Python imports are cheap however, so **why do you care**? – Martijn Pieters Aug 07 '18 at 08:47
  • When you look at any major Python framework (Django, Pandas, Qt, etc) you'll find that they *all have such namespaces*. This is not a problem you want to put too much energy into. :-) Note that most group multiple classes into modules; consider having just a `project/tickler.py` module file with both classes in it. – Martijn Pieters Aug 07 '18 at 08:49
  • @Amadan Well, wouldn't just that `__init__.py` do? Your loading logic will find the names through the package, even if their qualified names aren't exactly what was requested in the configuration. – AKX Aug 07 '18 at 08:50
  • @MartijnPieters: Exchange `kitten_tickler` with `tensorflow`, and it's not that cheap any more. The issue for me is file=module in the Python world, which bugs me, because in Java, directory=module; in Ruby, whatever you want=namespace, and thus both end up supporting this kind of structure where package name does not have to be redundant should you want a single class in a package. But Python apparently won't let me. :P And I know Python forces bundling multiple classes in a single file, but these might be a bit heavy so I don't like having them bundled. As I said, it's mostly aesthetics. – Amadan Aug 07 '18 at 08:52
  • So why fight it? Python is not Java and is not Ruby. You wouldn't try to make Java treat attributes and properties interchangeable either. – Martijn Pieters Aug 07 '18 at 08:53
  • Haha, no, you are completely right there. The point of the question is to find out if I am fighting windmills or if I am missing something where it is actually easy but I'm being stupid. By everyone's responses, I guess it's windmills after all. :) – Amadan Aug 07 '18 at 08:54
  • And if tensorflow classes are heavy (have a large loading cost), then that's certainly something that's worth mentioning in a question. Because then *perhaps* there are specific circumstances where you'd want to find a lazy loading scheme to provide a flat namespace without loading everything in the namespace at once. – Martijn Pieters Aug 07 '18 at 08:57
  • 1
    If you are using Python 3.7 you can load names in a module dynamically, see https://docs.python.org/3/whatsnew/3.7.html#whatsnew37-pep562 for the hooks. Then `project/ticklers/__init__.py` could load classes from nested modules only when actively accessed. I'd still advice you to not worry about the qualified name, even though you could mess with the `__module__` attribute. – Martijn Pieters Aug 07 '18 at 09:00
  • Okay, that's interesting, thank you for the reference. Unfortunately, I can't use 3.7 (`tensorflow`, err, I mean, `kitten_tickler`, isn't yet compatible with it) but I appreciate any time I learn something new - and `__getattr__` on modules is totally interesting (let alone relevant). (I'm already loading stuff dynamically, `importlib` is kind of a centerpiece... but I had no clue about the hooks.) – Amadan Aug 07 '18 at 09:03
  • `importlib` would let you define load a module into a custom module class that implements `__getattr__` too, but then you are really getting into 'here be dragons' territory. It's powerful though, so if you **really must** then it could be worth spending the engineering time on doing this (almost certainly with a 3.7 forward compatible method). – Martijn Pieters Aug 07 '18 at 10:35

0 Answers0