1

a few monthes ago, i installed mailman3 on clean installed debian 10 vm, with success – the server was working for long time and distributed hundreds of mails without any issues. The tutorial i implemented completely is that one announced by GNU Mailman itself.

After regular debian system upgrade process from 10 to 11 (didn't ever added external package sources, so it was really easy) i tried starting mailman 3 again. Everything works, except the django web framework – that one controlled by qcluster.service.

The problem seems to be within executing /opt/mailman/mm/bin/django-admin migrate; the output is as follows (journalctl -u qcluster.service contains the same lines):

Traceback (most recent call last):
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/utils.py", line 66, in __getitem__
    return self._engines[alias]
KeyError: 'django'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/backends/django.py", line 121, in get_package_libraries
    module = import_module(entry[1])
  File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/hyperkitty/templatetags/decorate.py", line 4, in <module>
    from hyperkitty.lib.renderer import markdown_renderer, text_renderer
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/hyperkitty/lib/renderer.py", line 7, in <module>
    from mistune.scanner import escape_html, escape_url
ImportError: cannot import name 'escape_html' from 'mistune.scanner' (/opt/mailman/mm/venv/lib/python3.9/site-packages/mistune/scanner.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/mailman/mm/venv/bin/django-admin", line 8, in <module>
    sys.exit(execute_from_command_line())
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/base.py", line 89, in wrapped
    res = handle_func(*args, **kwargs)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/commands/migrate.py", line 75, in handle
    self.check(databases=[database])
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/base.py", line 419, in check
    all_issues = checks.run_checks(
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/checks/registry.py", line 76, in run_checks
    new_errors = check(app_configs=app_configs, databases=databases)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/contrib/admin/checks.py", line 78, in check_dependencies
    for engine in engines.all():
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/utils.py", line 90, in all
    return [self[alias] for alias in self]
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/utils.py", line 90, in <listcomp>
    return [self[alias] for alias in self]
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/utils.py", line 81, in __getitem__
    engine = engine_cls(params)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/backends/django.py", line 25, in __init__
    options['libraries'] = self.get_templatetag_libraries(libraries)
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/backends/django.py", line 43, in get_templatetag_libraries
    libraries = get_installed_libraries()
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/backends/django.py", line 108, in get_installed_libraries
    for name in get_package_libraries(pkg):
  File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/backends/django.py", line 123, in get_package_libraries
    raise InvalidTemplateLibrary(
django.template.library.InvalidTemplateLibrary: Invalid template library specified. ImportError raised when trying to load 'hyperkitty.templatetags.decorate': cannot import name 'escape_html' from 'mistune.scanner' (/opt/mailman/mm/venv/lib/python3.9/site-packages/mistune/scanner.py)

output of pip freeze | grep django (be it that it helps):

django-allauth==0.47.0
django-appconf==1.0.5
django-compressor==3.1
django-extensions==3.1.5
django-gravatar2==1.4.4
django-haystack==3.1.1
django-mailman3==1.3.7
django-picklefield==3.0.1
django-q==1.3.9
djangorestframework==3.13.1

I already made sure that there aren't any unsolved pip package dependencies, but that doesn't seems to be the problem... I already tried to recreate python env, tried different python versions (Python 3.7.12, 3.8.12, 3.9.2 and 3.9.9) but that all didn't work either, googled half the web (found this but nothing helped :-(

dw-loewe
  • 11
  • 2

1 Answers1

1

Mistune has moved several functions from scanner to util, see this commit.

So you have to change from mistune.scanner import escape_html, escape_url to from mistune.util import escape_html, escape_url in /opt/mailman/mm/venv/lib/python3.9/site-packages/hyperkitty/lib/renderer.py as the error message suggests.

PS: Use pip freeze | grep -i django, because it appears as Django.

PS2: Seems like other mailman3 users are as unhappy as you are about the mistune dependency, see their gitlab bugtracker. This also means that hyperkitty devs should be aware of the new stable mistune version (they had a dependency on the 2.0 release candidate after which the refactoring in question happened).

frafl
  • 51
  • 6