0

I am trying to build the documentation for my project (https://github.com/pyMatJ/S4Utils) using sphinx and autodoc, targeting RtD eventually (https://s4utils.readthedocs.io/en/latest/).

So far I managed to make the whole project installable and I think my setup.py is in order. However, I cannot get sphinx to properly generate the documentation with autodoc. I set up a clean python3.6 venv using virtualenvwrapper, installed only sphinx and my package and tried to compile the doc (make html). The pages with manual input (installation etc.) work fine, as well as one autodocumented module (https://s4utils.readthedocs.io/en/latest/epsInterpolator.html). However, the bulk of the autodoc job is still missing (two other modules should be documented), and I get a bunch of warnings about autodoc not being able to import the module. Eventually it looks like it boils down to two main errors:

hbar_eV = h_eV/(2*np.pi)
TypeError: unsupported operand type(s) for *: 'int' and 'Mock'

from one module importing numpy in its preamble and using it to define a variable

as well as

numpy.core.multiarray failed to import for both modules.

The errors are reproduced both from RtD and a clean venv. I checked that only one version of numpy is installed (latest, 1.19.1). I use a conf.py file importing mock, and mocking the relevant (I suppose) modules:

from unittest.mock import Mock as MagicMock
MOCK_MODULES = ['numpy', 'matplotlib.pyplot', 'matplotlib.gridspec', 'matplotlib.widgets', 'pyvista']
for mod_name in MOCK_MODULES:
    sys.modules[mod_name] = MagicMock()

I add the files path to sys.path to allow autodoc to find them relatively to the docs directory (sys.path.insert(0, os.path.abspath('../../')))

  1. Regarding the failed numpy import: I am a bit puzzled since it is correctly handled in one of the autodocumented files. So why does this import fail for only two out of three (they all have the same import numpy as np statement in their preamble) ? I spent a few hours looking for a similar problem, but only found answers regarding conflicting numpy installations, which I believe I do not have. How do I fix this ?

  2. Regarding the Mock error, I tried to follow the lines of https://github.com/seb-m/pyinotify/issues/110 and redefined a few functions for mock, trying to make it accept the multiplication operation:

class Mock(MagicMock):
    @classmethod
    def __getattr__(cls, name):
        return Mock()
    def __mul__(self, other):
        return Mock()
    def __rmul__(self, other):
        return Mock()
    def __pow__(self, other):
        return Mock()
    def __div__(self, other):
        return Mock()

without any effect. I could get rid of the error by placing this operation in a class or defining it localy where needed, but it is rather anoying this this defines a physical constant that might be used in several classes/functions throughout the module later on. How do I make mock accept the multiplication ?

In case this might help, here is the relevant part of the directory structure:

--moduleroot/
  |--docs/
  |   |--requirements.txt
  |   |--Makefile
  |   |--make.bat
  |   |--source/
  |   |  |--conf.py
  |   |  |--rstfiles-for-doc
  |   |--build/
  |--module/
  |   |--__init__.py (probably useless?)
  |   |--Module1.py
  |   |--Module2.py
  |   |--Module3.py
  |--setup.py
  |--Manifest.in

Thank you in advance, any help would be appreciated at this point since I have no idea what I am doing wrong... (and my apologies if this is too long, since I have no idea of what goes wrong I tried to give as much -relevant- information as possible).

mzjn
  • 48,958
  • 13
  • 128
  • 248
pyMatJ
  • 3
  • 2
  • Does it make any difference if you use `autodoc_mock_imports`? See https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html#confval-autodoc_mock_imports – mzjn Jul 22 '20 at 15:26
  • Indeed ! This solves the `numpy.core.multiarray` bit, and subsequently clears out all other warnings I had related to it. The only remaining error concerns the mocked multiplication which changed a bit: `hbar_eV = h_eV/(2*np.pi) TypeError: unsupported operand type(s) for *: 'int' and 'pi'` – pyMatJ Jul 22 '20 at 16:33
  • 1
    I can reproduce that error, and it goes away if I use `math.pi` instead of `np.pi`. But I cannot explain why... – mzjn Jul 22 '20 at 17:04
  • It has to do with the mocking. If numpy is mocked, `np.pi` is not 3.14159.... – mzjn Jul 22 '20 at 20:36
  • Worked like a charm, both locally and on RtD. Thanks a lot ! I still find it slightly unsatisfying though. I expected `mock` to behave the same from the autodoc built-in function or manually, and I don't see why mocking a simple arithmetic operation is that complicated. Any pointer to get a clearer view on this is welcome ! I cannot yet upvote this, could you make an answer so I can accept it ? – pyMatJ Jul 22 '20 at 22:20

1 Answers1

1

This fixed the issues:

  • Use the autodoc_mock_imports configuration variable to specify modules to be mocked.

  • When NumPy is mocked, np.pi does not work. hbar_eV = h_eV/(2*np.pi) is module-level code that is executed every time Sphinx runs. To get around that, use math.pi instead.

mzjn
  • 48,958
  • 13
  • 128
  • 248