Understanding jinja2.Packageloader()

Question

I have a (mostly working) python3 program using mkepub.

Since I need to modify it for my specific purposes I started pulling it into my directory tree (I will send a Pull Request when I'm done since I think my mods could be useful to others, but this isn't the problem).

My current directory tree is:

.
├── markright
│   ├── book_parser.py
│   ├── docx_emitter.py
│   ├── emitter.py
│   ├── epub_emitter.py
│   ├── html_emitter.py
│   ├── __init__.py
│   ├── mkepub
│   │   ├── __init__.py
│   │   ├── mkepub.py
│   │   ├── templates
│   │   │   ├── container.xml
│   │   │   ├── cover.xhtml
│   │   │   ├── package.opf
│   │   │   ├── page.xhtml
│   │   │   ├── toc.ncx
│   │   │   └── toc.xhtml
│   │   └── tests
├── markright.py

I'm using mkepub from epub_emitter.py in this way:

from . import mkepub  # <-- here I get problems
...
epub = mkepub.Book(**kwargs)

which imports ./mkepub/__init__.py:

from .mkepub import Book  # <-- here I get problems

... which finally gets to problematic code:

import jinja2
...
env = jinja2.Environment(loader=jinja2.PackageLoader('mkepub'))

This bombs at program startup (while importing) with the following error:

Traceback (most recent call last):
  File "/home/mcon/Documents/Libro/sigil/markright/venv/lib/python3.7/site-packages/setuptools-40.8.0-py3.7.egg/pkg_resources/__init__.py", line 359, in get_provider
KeyError: 'mkepub'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mcon/Documents/Libro/sigil/markright/markright.py", line 1, in <module>
    from markright import *
  File "/home/mcon/Documents/Libro/sigil/markright/markright/__init__.py", line 4, in <module>
    from .epub_emitter import EPUBemitter
  File "/home/mcon/Documents/Libro/sigil/markright/markright/epub_emitter.py", line 1, in <module>
    from . import mkepub
  File "/home/mcon/Documents/Libro/sigil/markright/markright/mkepub/__init__.py", line 1, in <module>
    from .mkepub import Book
  File "/home/mcon/Documents/Libro/sigil/markright/markright/mkepub/mkepub.py", line 49, in <module>
    env = jinja2.Environment(loader=jinja2.PackageLoader('mkepub'))
  File "/home/mcon/Documents/Libro/sigil/markright/venv/lib/python3.7/site-packages/jinja2/loaders.py", line 224, in __init__
    provider = get_provider(package_name)
  File "/home/mcon/Documents/Libro/sigil/markright/venv/lib/python3.7/site-packages/setuptools-40.8.0-py3.7.egg/pkg_resources/__init__.py", line 361, in get_provider
ModuleNotFoundError: No module named 'mkepub'

Note this works if I import the same mkepub (I did not modify it, yet) from system directory (or venv).

What am I missing?

Of course I would like to change as little as possible (ideally nothing!) in mkepub to avoid problems in future Pull Request. OTOH I can change without any problem my code.

In the process I would also like to understand better path resolution in Python3.

UPDATE: apparently the only way is to install mkepub in a venv and then edit files inside it (.../venv/lib/python3.7/site-packages/mkepub/...) which looks very ugly. Is there another way?

I would suggest not trying to embed the `mkepub` module *inside* your own module, but instead just install it as a requirement. Then you will be able to `import mkepub`, and the `PackageLoader` call will find it where it expects to find it. — larsks, Jun 02 '19 at 14:07
Thanks @larsks, I think I see what You mean (not sure, since I *will* need to modify `mkepub`, so I *will* have a `mkepub` directory *beside* my dir if not *inside* it), but I would really like to understand what's going on, not just fix it. I'll try to pull `mkepub` alongside my `markright` dir. — ZioByte, Jun 02 '19 at 14:39

score 0 · Answer 1 · answered May 03 '23 at 11:59

You have a markright module defined as a folder and another one defined as a markright.py file. That's a first problem. If you defined markright.py to use it as an entry point, you should use instead:

python -m markright

which will run the "main.py" file within the "markright" folder, which you need to create of course.

From "markright/mkepub/mkepub.py", you must call "PackageLoader" with argument "markright.mkepub", since it is the name of your package. It will be configured to load files from "markright/mkepub/templates".

Finally if you need a custom version of mkepub, you should still work on it separately. Clone the online git repository to have the entire source code, make your changes, use pip or any other applicable packing tool to build the binary distributable as a wheel file, and you will be able to install your own version either globally (not recommended) or locally (in a venv) using "pip install" with the name of the wheel file. Take care of changing the version number (the minimum would be a "patch number").

It is perfectly viable to have a custom mkepub installed locally in your project's venv. While you work on mkepub to add your changes, you can also share the venv (you can put venvs anywhere) with the markright project. It is not essential but will ensure compatibility. If you go along this path, of course, mkepub will be a top level package and "PackageLoader" will only need "mkepub".

Understanding jinja2.Packageloader()

1 Answers1