68

I have a package mypack with modules mod_a and mod_b in it. I intend the package itself and mod_a to be imported freely:

import mypack
import mypack.mod_a

However, I'd like to keep mod_b for the exclusive use of mypack. That's because it exists merely to organize the latter's internal code.

My first question is, is it an accepted practice in Python programming to have 'private' modules like this?

If yes, my second question is, what is the best way to convey this intention to the client? Do I prefix the name with an underscore (i.e. _mod_b)? Or would it be a good idea to declare a sub-package private and place all such modules there?

G S
  • 35,511
  • 22
  • 84
  • 118

5 Answers5

60

I prefix private modules with an underscore to communicate the intent to the user. In your case, this would be mypack._mod_b

This is in the same spirit (but not completely analogous to) the PEP8 recommendation to name C-extension modules with a leading underscore when it’s wrapped by a Python module; i.e., _socket and socket.

Zearin
  • 1,474
  • 2
  • 17
  • 36
Jeremy
  • 1,397
  • 2
  • 13
  • 20
  • 8
    Actually [PEP8](https://www.python.org/dev/peps/pep-0008/?#public-and-internal-interfaces) says: `Even with __all__ set appropriately, internal interfaces (packages, modules, classes, functions, attributes or other names) should still be prefixed with a single leading underscore.` – aviso Jun 26 '18 at 20:22
19

The solution I've settled on is to create a sub-package 'private' and place all the modules I wish to hide in there. This way they stay stowed away, leaving mypack's module list cleaner and easier to parse.

To me, this doesn't look unpythonic either.

G S
  • 35,511
  • 22
  • 84
  • 118
  • 1
    I am coming to this post nearly 6 years later with the same questions and motivations that you had. Is the private sub-package approach still the one you take or have you tried other approaches since then? – sparc_spread Jun 29 '16 at 21:59
  • 4
    Yes. I'm still using this method. Haven't found any better alternative. – G S Jul 07 '16 at 05:45
  • 4
    Thanks for update and for suggesting this approach in the first place. I prefix the "private" sub-package name with a 'z_', so that it appears last in IDE code-completion dropdowns. I use relative imports in the sub-package's `__init__.py` to expose only the "public" functions. In PyCharm and Jupyter, to keep the names of the internal modules out of dropdowns, each module must have a function with the same name as the module, and I must import the function in `__init__.py`. You probably know all this already but if not here it is. If you want me to expand as an answer just let me know. – sparc_spread Jul 07 '16 at 15:16
  • 8
    If I saw `private` as a module name I would be left wondering what's in the package. I wouldn't like to have to read through the contents to work out why it's organised as such. Module and package names should indicate what's in them, rather than their private or public status – joel Sep 30 '19 at 22:47
11

While there are not explicit private keywords there is a convention to have put private functions start with a single underscore but a double leading underscore will make it so others cannot easily call the function from outside the module. See the following from PEP 8

- _single_leading_underscore: weak "internal use" indicator.  E.g. "from M
  import *" does not import objects whose name starts with an underscore.

- single_trailing_underscore_: used by convention to avoid conflicts with
  Python keyword, e.g.

  Tkinter.Toplevel(master, class_='ClassName')

- __double_leading_underscore: when naming a class attribute, invokes name
  mangling (inside class FooBar, __boo becomes _FooBar__boo; see below).

- __double_leading_and_trailing_underscore__: "magic" objects or
  attributes that live in user-controlled namespaces.  E.g. __init__,
  __import__ or __file__.  Never invent such names; only use them
  as documented.

To make an entire module private, don't include it __init__.py file.

aterrel
  • 1,992
  • 1
  • 14
  • 23
  • 9
    Aterrel, what do you mean by "don't include in `__init__.py`"? Do mean don't place the statement `import mod_b` in `__init__.py`? If so, that still doesn't solve my problem. The client can still import `mod_b` just like a public module: `import mypack.mod_b`. Or do you mean something else? – G S Aug 31 '10 at 16:44
  • 2
    So it won't strictly enforce the module being private, just as a single underscore is weak. But it doesn't show up without explicitly importing it, which is still pretty private. I imagine you could name the file with two leading underscores, but I haven't ever played with this. – aterrel Sep 01 '10 at 03:17
  • 2
    Thanks for the response aterrel. The solution I'm settling on is creating a subpackage `private` under `mypack` and placing all modules I want to 'hide' in that. This serves my purpose of making my code easier to explore (through intellisense, for example), by tucking away all the uneeded fluff. – G S Sep 02 '10 at 06:20
6

One thing to be aware of in this scenario is indirect imports. If in mypack you

from mypack._mod_b import foo
foo()

Then a user can

from mypack import foo
foo()

and be none the wiser. I recommend importing as

from mypack import _mod_b
_mod_b.foo()

then a user will immediately see a red flag when they try to

from mypack import _mod_b

As for actual directory structure, you could even extend Jeremy's answer into a _package_of_this_kind package, where anything in that can have any 'access modifiers' on it you like - users will know there be dragons

joel
  • 6,359
  • 2
  • 30
  • 55
2

Python doesn't strictly know or support "private" or "protected" methods or classes. There's a convention that methods prefixed with a single underscore aren't part of an official API, but I wouldn't do this on classes or files - it's ugly.

If someone really needs to subclass or access mod_b, why prevent him/her from doing so? You can always supply a preferred API in your documentation and document in your module that you shouldn't access it directly and use mypack in stead.

Ivo van der Wijk
  • 16,341
  • 4
  • 43
  • 57
  • 8
    I could leave `mod_b` accessible just like `mod_a`. However, imagine there are 20 modules akin to `mod_b`. So when the client types `mypack.` in the editor, the intellisense will list all 25 or so private as well as public modules. This will make my code harder to explore. That's the reason I want to 'hide' those modules which are not meant for the client. As for documentation, people generally prefer exploring to reading documentation. – G S Aug 31 '10 at 17:22
  • As Matt Joiner said, don't import `mod_b` into your `__init__.py`. I suggest you also look up the `__all__` special variable. If neither of those gets the job done, the problem is in Intellisense. – ssokolow Sep 04 '10 at 21:33
  • 4
    ssokolow, see the third comment under Matt's answer. Even when `mod_b` is not imported in `mypack/__init__.py`, typing `import mypack.mod_b` in the client code still works successfully. That's because Python doesn't require importing of nested modules into top level ones for "dot" imports (like `mypack.mod_b`) to work. – G S Sep 05 '10 at 06:24
  • 3
    "If someone really needs to subclass or access mod_b" this is precisely the thinking behind using underscores - they inform clients what is and what isn't part of the API – joel Sep 30 '19 at 22:32
  • Because I don't want to read your documentation xD I just want to know what's the API, code aesthetics sure it's important, but more important is for me to get the information ASAP without having to read documentation – José Rodrigues Oct 01 '22 at 12:22