MkDocs with mkdocstrings fails to serve with "Could not collect" error

Question

I am using the Python MkDocs tool to build API documentation extracted directly from the source using the MkDocs mkdocstrings extension.

When I run the mkdocs serve command inside my project (in the directory which contains the mkdocs.yml file), the command fails with a ERROR - Could not collect 'cpinotify' error.

aUser@aMachine:~/notify2nats$ mkdocs serve
INFO     -  Building documentation...
INFO     -  Cleaning site directory
ERROR    -  mkdocstrings.extension: Error while loading JSON:

            Traceback (most recent call last):
              File
            "/home/aUser/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/mkdocstrings/handlers/python.py",
            line 206, in collect
                result = json.loads(stdout)
              File
            "/home/aUser/.asdf/installs/python/3.9.6/lib/python3.9/json/__init__.py",
            line 346, in loads
                return _default_decoder.decode(s)
              File
            "/home/aUser/.asdf/installs/python/3.9.6/lib/python3.9/json/decoder.py",
            line 337, in decode
                obj, end = self.raw_decode(s, idx=_w(s, 0).end())
              File
            "/home/aUser/.asdf/installs/python/3.9.6/lib/python3.9/json/decoder.py",
            line 355, in raw_decode
                raise JSONDecodeError("Expecting value", s, err.value) from None
            json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
ERROR    -  Error reading page 'API.md':
ERROR    -  Could not collect 'cpinotify'

Aborted with a BuildError!

My docs/API.md file which contains the MkDocs mkdocstrings command is:

# Test

::: cpinotify

(Where cpinotify is the name of the Python package from which the mkdocstrings documentation is to be extracted).

Versions:

Python v3.9.6
MkDocs v1.2.2
mkdocstrings v0.15.2
Ubuntu 20.04
asdf v0.8.1-a1ef92a (to virtualize Python versions)
pipx v0.16.3 (to virtualize installed Python packages/commands)

score 3 · Answer 1 · answered Jul 25 '21 at 07:14

TL;DR Do not use the Python argparser at the top level in any module.

After some work (hacking the mkdocstrings code to understand what was happening)... I identified that (in my case) the problem was that my cpinotify code was using the Python argparse library at the top level in the cpinotify/cpinotify.py module.

The mkdocstrings code uses the pytkdocs command to extract the information it will use to build the documentation from the relevant Python code. The mkdocstrings code runs pytkdocs as an external process, feeding the required package information as JSON on pytkdocs stdin, and expects the results as JSON on pytkdocs stdout.

pytkdocs actually imports the code to be documented. Since both my cpinotify code as well as pytkdocs uses the argparse library, our two uses were conflicting and resulting in an error message and no JSON for the mkdocstrings extension to load.

While the Python argparse documentation shows examples where the argparser is created at the top level, this use at the top level when imported by pytkdocs was changing the pytkdocs expected command line options, causing pytkdocs to fail with no output.

Solution: Once I moved my use of the argparse library inside the cpinotify.cli method, my use of argparse was no longer being called when imported, and so was no longer changing pytkdocs expected command line arguments.

Having made this change to my code, the pktdocs command started working as expected and the mkdocs serve command worked with no errors producing the documentation I expected.

Note: the actual errors in my question (above) came from the fact that pytkdocs had failed to provide any output -- so mkdocstrings will provide similar failure reports any time that pytkdocs fails with no output.

score 1 · Answer 2 · answered Mar 21 '22 at 12:31

This has been fixed as of mkdocstrings[python]>=0.18, which uses griffe as its docs collector instead of pytkdocs, and by default does not execute the code being documented.

Migration notes: https://mkdocstrings.github.io/handlers/overview/#migrate-to-the-experimental-python-handler

Changelog: https://mkdocstrings.github.io/changelog/#0180-2022-02-06

Griffe and mkdocstrings 0.18 still don't (as of 2022-03-21) support all of the features that the older versions of mkdocstrings supported, such as element filtering. That said, the newer version come with numerous advantages, so I would recommend migrating over if possible now.

MkDocs with mkdocstrings fails to serve with "Could not collect" error

2 Answers2