genindex and modindex footer links don't work in readthedocs.io

Question

I have a Python project using Sphinx for docs. I am building the docs remotely on readthedocs.io service.

I used sphinx-quickstart and it generated an index.rst file with these links in the footer:

Indices and tables
~~~~~~~~~~~~~~~~~~

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

When I push changes to readthedocs.io and build the docs, my build succeeds. Docs that I manually linked via toctree directive all work fine.

The search link works fine.

But the genindex link goes to an empty page, titled "Index"

And the modindex page links to py-modindex.html, which is a 404.

Following this guide: https://samnicholls.net/2016/06/15/how-to-sphinx-readthedocs I had run sphinx-apidoc -o api-docs/ ../myproject to generate the autodoc .rst files.

I linked the resulting api-docs/modules.rst in the toctree section at the top of my index.rst... That link works and if I click through the api-docs have been generated correctly.

Also generated by sphinx-autodoc were files for each package in my project, they contain directives like:

myproject.whatever module
-------------------------

.. automodule:: myproject.whatever
   :members:
   :undoc-members:
   :show-inheritance:

If I browse directly to these pages they have content, but they don't appear in the index (only the tocs they are manually linked in).

I also have some manually authored pages, again linked via toc.

My docs/conf.py looks like:

import os
import sys

sys.path.insert(0, os.path.abspath("../"))

extensions = [
    "sphinx.ext.autodoc",
    "sphinx.ext.viewcode",
    "sphinx.ext.napoleon",
    "sphinx_autodoc_typehints",
]

templates_path = ["_templates"]

exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]

html_theme = "alabaster"

html_static_path = ["_static"]

I believe the fact that html generated from the autodoc .rst stub files are filled with modules and classes extracted from the .py files in my project indicates that the sys path fix and autodoc are basically working.

So my question is:

• How to make :ref:`genindex` have some content?
• How to fix :ref:`modindex` points to py-modindex.html which does not exist?

Answer 1

genindex and modindex are automatically managed by Sphinx. Two cases should be considered:

1. Any declaration in a .rst file will be inserted in those indexes. For example, if you declare a function from the Python domain:

Your rst file
-------------

.. py:function:: name(parameters)

It will be inserted in the indexes even if it doesn't have a corresponding function in any .py file.

2. Using autodoc directives, the same applies with a few more rules. The autodoc extension will substitute domain declarations (like above) depending if the object has a docstring and if you are using the :members: and or :undoc-members: options. So you have to verify you are using the correct option and directive for your case.

Your rst file
-------------

.. autoclass:: Your_Module.Your_Class
   :members:

The above example will be substituted by a :py:class:: domain declaration if the corresponding class has a docstring, if not you need to add the :undoc-members: option.

---

---

The symptoms you are describing of having empty indexes happens when you haven't declared anything in the .rst files. With the nuance that the autodoc directives may or may not do those declarations for you depending if the objects have docstrings and you used the right options in the directives.

---

---

EDIT: You should also run make clean before your builds (e.g.make html) because inconsistencies between builds can break the index.

Answer 2

As I eventually worked out in the comments, thanks to help from @bad_coder, my problem was specific to building the docs in readthedocs.io

Building the docs locally worked fine.

The reason came down to use of sphinx.ext.autodoc, perhaps in conjunction with sphinx_autodoc_typehints, which seems to need to import my actual python code. Checking the logs of my apparently successful readthedocs build showed actually there were warnings like:

WARNING: autodoc: failed to import module 'whatever' from module 'myproject'; the following exception was raised:
No module named 'somelib'

i.e the docs had only partially built, it had skipped the parts it couldn't do.

The build worked locally because I was already in a virtualenv with all my project's dependencies installed.

(IMHO this seems like a bad design of the sphinx.ext.autodoc and/or sphinx_autodoc_typehints ...good static-analysis tools exist for Python which can build an AST or CST and extract structure and docstrings without importing any code.)

Well anyway, this meant that I needed to tell readthedocs.io how to install all my project deps. This is slightly complicated by the fact I'm using Poetry, which is not explicitly supported. This means I don't have a requirements.txt file to point to (and I don't want to manually create one that is a duplicate of everything in my pyproject.toml).

Fortunately the pyproject.toml file is understandable by pip, so we're able to use the pip install method described here for readthedocs.io to install both my project deps, plus extra deps that are only needed for building docs: https://github.com/readthedocs/readthedocs.org/issues/4912#issuecomment-664002569

To summarise:

1. Deleted my docs/requirements.txt
2. Added:

   [tool.poetry.dependencies]
   ...
   sphinx = {version = "^3.1.1", optional = true}
   sphinx-autodoc-typehints ={version = "^1.11.1", optional = true}
   

   and:

   [tool.poetry.extras]
   docs = [
       "sphinx",
       "sphinx-autodoc-typehints"
   ]
   

   to my pyproject.toml
3. Updated my .readthedocs.yml to:

   version: 2
   
   sphinx:
     configuration: docs/conf.py
   
   python:
     version: 3.8
     install:
       - method: pip
         path: .
         extra_requirements:
           - docs
   

4. Pushed these changes to readthedocs.io ...voilà, now it works.