Generating documentation for flask extensions using sphinx

Question

I am attempting to generate documentation for flask-restfulan extension for flask that makes it easy to generate RESTful APIs. My final goal is to use doc2dash to generate dash doctests.

However, before I can do this, I need to generate the html required using sphinx. In my attempts to do so I have cloned the flask-restful project, and have tried to generate the docs from its docs folder, using make html. However, I get 17 warning, and the resulting documentation is incomplete. Here is the traceback:

sphinx-build -b html -d _build/doctrees   . _build/html
Making output directory...
Running Sphinx v1.2.3
loading pickled environment... not yet created
loading intersphinx inventory from http://flask.pocoo.org/docs/objects.inv...
building [html]: targets for 9 source files that are out of date
updating environment: 9 added, 0 changed, 0 removed
reading sources... [100%] testing
/Users/quazinafiulislam/Code/Python/restful/docs/api.rst:9: WARNING: autodoc: failed to import function u'marshal' from module u'flask.ext.restful'; the following exception was raised:
Traceback (most recent call last):
  File "/Users/quazinafiulislam/.pyenv/versions/2.7.6/lib/python2.7/site-packages/sphinx/ext/autodoc.py", line 335, in import_object
    __import__(self.modname)
  File "/Users/quazinafiulislam/.pyenv/versions/2.7.6/lib/python2.7/site-packages/flask/exthook.py", line 87, in load_module
    raise ImportError('No module named %s' % fullname)
ImportError: No module named flask.ext.restful
/Users/quazinafiulislam/Code/Python/restful/docs/api.rst:10: WARNING: autodoc: failed to import function u'marshal_with' from module u'flask.ext.restful'; the following exception was raised:
Traceback (most recent call last):
  File "/Users/quazinafiulislam/.pyenv/versions/2.7.6/lib/python2.7/site-packages/sphinx/ext/autodoc.py", line 335, in import_object
    __import__(self.modname)
  File "/Users/quazinafiulislam/.pyenv/versions/2.7.6/lib/python2.7/site-packages/flask/exthook.py", line 87, in load_module
    raise ImportError('No module named %s' % fullname)
ImportError: No module named flask.ext.restful
/Users/quazinafiulislam/Code/Python/restful/docs/api.rst:11: WARNING: autodoc: failed to import function u'marshal_with_field' from module u'flask.ext.restful'; the following exception was raised:
Traceback (most recent call last):
  File "/Users/quazinafiulislam/.pyenv/versions/2.7.6/lib/python2.7/site-packages/sphinx/ext/autodoc.py", line 335, in import_object
    __import__(self.modname)
  File "/Users/quazinafiulislam/.pyenv/versions/2.7.6/lib/python2.7/site-packages/flask/exthook.py", line 87, in load_module
    raise ImportError('No module named %s' % fullname)
ImportError: No module named flask.ext.restful
/Users/quazinafiulislam/Code/Python/restful/docs/api.rst:12: WARNING: autodoc: failed to import function u'abort' from module u'flask.ext.restful'; the following exception was raised:
Traceback (most recent call last):
  File "/Users/quazinafiulislam/.pyenv/versions/2.7.6/lib/python2.7/site-packages/sphinx/ext/autodoc.py", line 335, in import_object
    __import__(self.modname)
  File "/Users/quazinafiulislam/.pyenv/versions/2.7.6/lib/python2.7/site-packages/flask/exthook.py", line 87, in load_module
    raise ImportError('No module named %s' % fullname)
ImportError: No module named flask.ext.restful
/Users/quazinafiulislam/Code/Python/restful/docs/api.rst:17: WARNING: autodoc: failed to import class u'Api' from module u'flask.ext.restful'; the following exception was raised:
Traceback (most recent call last):
  File "/Users/quazinafiulislam/.pyenv/versions/2.7.6/lib/python2.7/site-packages/sphinx/ext/autodoc.py", line 335, in import_object
    __import__(self.modname)
  File "/Users/quazinafiulislam/.pyenv/versions/2.7.6/lib/python2.7/site-packages/flask/exthook.py", line 87, in load_module
    raise ImportError('No module named %s' % fullname)
ImportError: No module named flask.ext.restful
/Users/quazinafiulislam/Code/Python/restful/docs/api.rst:22: WARNING: autodoc: failed to import class u'Resource' from module u'flask.ext.restful'; the following exception was raised:
Traceback (most recent call last):
  File "/Users/quazinafiulislam/.pyenv/versions/2.7.6/lib/python2.7/site-packages/sphinx/ext/autodoc.py", line 335, in import_object
    __import__(self.modname)
  File "/Users/quazinafiulislam/.pyenv/versions/2.7.6/lib/python2.7/site-packages/flask/exthook.py", line 87, in load_module
    raise ImportError('No module named %s' % fullname)
ImportError: No module named flask.ext.restful
/Users/quazinafiulislam/Code/Python/restful/docs/api.rst:27: WARNING: autodoc: failed to import module u'reqparse'; the following exception was raised:
Traceback (most recent call last):
  File "/Users/quazinafiulislam/.pyenv/versions/2.7.6/lib/python2.7/site-packages/sphinx/ext/autodoc.py", line 335, in import_object
    __import__(self.modname)
  File "/Users/quazinafiulislam/Code/Python/restful/flask_restful/reqparse.py", line 3, in <module>
    import flask_restful
ImportError: No module named flask_restful
/Users/quazinafiulislam/Code/Python/restful/docs/api.rst:30: WARNING: autodoc: failed to import class u'Argument' from module u'flask.ext.restful'; the following exception was raised:
Traceback (most recent call last):
  File "/Users/quazinafiulislam/.pyenv/versions/2.7.6/lib/python2.7/site-packages/sphinx/ext/autodoc.py", line 335, in import_object
    __import__(self.modname)
  File "/Users/quazinafiulislam/.pyenv/versions/2.7.6/lib/python2.7/site-packages/flask/exthook.py", line 87, in load_module
    raise ImportError('No module named %s' % fullname)
ImportError: No module named flask.ext.restful
/Users/quazinafiulislam/Code/Python/restful/docs/api.rst:37: WARNING: autodoc: failed to import module u'fields'; the following exception was raised:
Traceback (most recent call last):
  File "/Users/quazinafiulislam/.pyenv/versions/2.7.6/lib/python2.7/site-packages/sphinx/ext/autodoc.py", line 335, in import_object
    __import__(self.modname)
  File "/Users/quazinafiulislam/Code/Python/restful/flask_restful/fields.py", line 9, in <module>
    from flask_restful import inputs, marshal
ImportError: No module named flask_restful
/Users/quazinafiulislam/Code/Python/restful/docs/api.rst:41: WARNING: Title underline too short.

Inputs
-----
/Users/quazinafiulislam/Code/Python/restful/docs/api.rst:41: WARNING: Title underline too short.

Inputs
-----
/Users/quazinafiulislam/Code/Python/restful/docs/api.rst:44: WARNING: autodoc: failed to import function u'url' from module u'flask.ext.restful.inputs'; the following exception was raised:
Traceback (most recent call last):
  File "/Users/quazinafiulislam/.pyenv/versions/2.7.6/lib/python2.7/site-packages/sphinx/ext/autodoc.py", line 335, in import_object
    __import__(self.modname)
  File "/Users/quazinafiulislam/.pyenv/versions/2.7.6/lib/python2.7/site-packages/flask/exthook.py", line 87, in load_module
    raise ImportError('No module named %s' % fullname)
ImportError: No module named flask.ext.restful
/Users/quazinafiulislam/Code/Python/restful/docs/api.rst:45: WARNING: autodoc: failed to import function u'date' from module u'flask.ext.restful.inputs'; the following exception was raised:
Traceback (most recent call last):
  File "/Users/quazinafiulislam/.pyenv/versions/2.7.6/lib/python2.7/site-packages/sphinx/ext/autodoc.py", line 335, in import_object
    __import__(self.modname)
  File "/Users/quazinafiulislam/.pyenv/versions/2.7.6/lib/python2.7/site-packages/flask/exthook.py", line 87, in load_module
    raise ImportError('No module named %s' % fullname)
ImportError: No module named flask.ext.restful
/Users/quazinafiulislam/Code/Python/restful/docs/api.rst:46: WARNING: autodoc: failed to import function u'iso8601interval' from module u'flask.ext.restful.inputs'; the following exception was raised:
Traceback (most recent call last):
  File "/Users/quazinafiulislam/.pyenv/versions/2.7.6/lib/python2.7/site-packages/sphinx/ext/autodoc.py", line 335, in import_object
    __import__(self.modname)
  File "/Users/quazinafiulislam/.pyenv/versions/2.7.6/lib/python2.7/site-packages/flask/exthook.py", line 87, in load_module
    raise ImportError('No module named %s' % fullname)
ImportError: No module named flask.ext.restful
/Users/quazinafiulislam/Code/Python/restful/docs/api.rst:47: WARNING: autodoc: failed to import function u'natural' from module u'flask.ext.restful.inputs'; the following exception was raised:
Traceback (most recent call last):
  File "/Users/quazinafiulislam/.pyenv/versions/2.7.6/lib/python2.7/site-packages/sphinx/ext/autodoc.py", line 335, in import_object
    __import__(self.modname)
  File "/Users/quazinafiulislam/.pyenv/versions/2.7.6/lib/python2.7/site-packages/flask/exthook.py", line 87, in load_module
    raise ImportError('No module named %s' % fullname)
ImportError: No module named flask.ext.restful
/Users/quazinafiulislam/Code/Python/restful/docs/api.rst:48: WARNING: autodoc: failed to import function u'boolean' from module u'flask.ext.restful.inputs'; the following exception was raised:
Traceback (most recent call last):
  File "/Users/quazinafiulislam/.pyenv/versions/2.7.6/lib/python2.7/site-packages/sphinx/ext/autodoc.py", line 335, in import_object
    __import__(self.modname)
  File "/Users/quazinafiulislam/.pyenv/versions/2.7.6/lib/python2.7/site-packages/flask/exthook.py", line 87, in load_module
    raise ImportError('No module named %s' % fullname)
ImportError: No module named flask.ext.restful
/Users/quazinafiulislam/Code/Python/restful/docs/api.rst:49: WARNING: autodoc: failed to import function u'rfc822' from module u'flask.ext.restful.inputs'; the following exception was raised:
Traceback (most recent call last):
  File "/Users/quazinafiulislam/.pyenv/versions/2.7.6/lib/python2.7/site-packages/sphinx/ext/autodoc.py", line 335, in import_object
    __import__(self.modname)
  File "/Users/quazinafiulislam/.pyenv/versions/2.7.6/lib/python2.7/site-packages/flask/exthook.py", line 87, in load_module
    raise ImportError('No module named %s' % fullname)
ImportError: No module named flask.ext.restful
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] testing
writing additional files... genindex search
copying static files... done
copying extra files... done
dumping search index... done
dumping object inventory... done
build succeeded, 17 warnings.

Build finished. The HTML pages are in _build/html.

From looking at the results, one can see that the problem is that in the documentation the flask-restful extension is referred to ask flask.ext.restful. However, since the generation of this documentation requires the source-code, sphinx is unable to find it and hence raises and ImportError, because the module is called flask_restful. Here is that the tree looks like:

.
├── docs
│   ├── _build
│   ├── _static
│   ├── _templates
│   └── _themes
│       ├── flask
│       │   └── static
│       └── flask_small
│           └── static
├── examples
├── flask_restful  <---------- Not flask.ext.restful
│   ├── representations
│   └── utils
├── scripts
└── tests

So, the question is how do I get sphinx to understand that flask.ext.restful is actually flask_restful?

Update, after following Antti's advice, this is what it comes to:

rm -rf _build/*
sphinx-build -b html -d _build/doctrees   . _build/html
Making output directory...
Running Sphinx v1.2.3
loading pickled environment... not yet created
loading intersphinx inventory from http://flask.pocoo.org/docs/objects.inv...
building [html]: targets for 9 source files that are out of date
updating environment: 9 added, 0 changed, 0 removed
reading sources... [100%] testing
/Users/quazinafiulislam/Code/Python/restful/docs/api.rst:11: WARNING: autodoc: failed to import function u'marshal_with_field' from module u'flask.ext.restful'; the following exception was raised:
Traceback (most recent call last):
  File "/Users/quazinafiulislam/Code/Python/restful/.venv/lib/python2.7/site-packages/sphinx/ext/autodoc.py", line 342, in import_object
    obj = self.get_attr(obj, part)
  File "/Users/quazinafiulislam/Code/Python/restful/.venv/lib/python2.7/site-packages/sphinx/ext/autodoc.py", line 241, in get_attr
    return safe_getattr(obj, name, *defargs)
  File "/Users/quazinafiulislam/Code/Python/restful/.venv/lib/python2.7/site-packages/sphinx/util/inspect.py", line 114, in safe_getattr
    raise AttributeError(name)
AttributeError: marshal_with_field
/Users/quazinafiulislam/Code/Python/restful/docs/api.rst:37: WARNING: autodoc: failed to import module u'fields'; the following exception was raised:
Traceback (most recent call last):
  File "/Users/quazinafiulislam/Code/Python/restful/.venv/lib/python2.7/site-packages/sphinx/ext/autodoc.py", line 335, in import_object
    __import__(self.modname)
  File "/Users/quazinafiulislam/Code/Python/restful/flask_restful/fields.py", line 9, in <module>
    from flask_restful import inputs, marshal
ImportError: cannot import name inputs
/Users/quazinafiulislam/Code/Python/restful/docs/api.rst:41: WARNING: Title underline too short.

Inputs
-----
/Users/quazinafiulislam/Code/Python/restful/docs/api.rst:41: WARNING: Title underline too short.

Inputs
-----
/Users/quazinafiulislam/Code/Python/restful/docs/api.rst:44: WARNING: autodoc: failed to import function u'url' from module u'flask.ext.restful.inputs'; the following exception was raised:
Traceback (most recent call last):
  File "/Users/quazinafiulislam/Code/Python/restful/.venv/lib/python2.7/site-packages/sphinx/ext/autodoc.py", line 335, in import_object
    __import__(self.modname)
  File "/Users/quazinafiulislam/Code/Python/restful/.venv/lib/python2.7/site-packages/flask/exthook.py", line 87, in load_module
    raise ImportError('No module named %s' % fullname)
ImportError: No module named flask.ext.restful.inputs
/Users/quazinafiulislam/Code/Python/restful/docs/api.rst:45: WARNING: autodoc: failed to import function u'date' from module u'flask.ext.restful.inputs'; the following exception was raised:
Traceback (most recent call last):
  File "/Users/quazinafiulislam/Code/Python/restful/.venv/lib/python2.7/site-packages/sphinx/ext/autodoc.py", line 335, in import_object
    __import__(self.modname)
  File "/Users/quazinafiulislam/Code/Python/restful/.venv/lib/python2.7/site-packages/flask/exthook.py", line 87, in load_module
    raise ImportError('No module named %s' % fullname)
ImportError: No module named flask.ext.restful.inputs
/Users/quazinafiulislam/Code/Python/restful/docs/api.rst:46: WARNING: autodoc: failed to import function u'iso8601interval' from module u'flask.ext.restful.inputs'; the following exception was raised:
Traceback (most recent call last):
  File "/Users/quazinafiulislam/Code/Python/restful/.venv/lib/python2.7/site-packages/sphinx/ext/autodoc.py", line 335, in import_object
    __import__(self.modname)
  File "/Users/quazinafiulislam/Code/Python/restful/.venv/lib/python2.7/site-packages/flask/exthook.py", line 87, in load_module
    raise ImportError('No module named %s' % fullname)
ImportError: No module named flask.ext.restful.inputs
/Users/quazinafiulislam/Code/Python/restful/docs/api.rst:47: WARNING: autodoc: failed to import function u'natural' from module u'flask.ext.restful.inputs'; the following exception was raised:
Traceback (most recent call last):
  File "/Users/quazinafiulislam/Code/Python/restful/.venv/lib/python2.7/site-packages/sphinx/ext/autodoc.py", line 335, in import_object
    __import__(self.modname)
  File "/Users/quazinafiulislam/Code/Python/restful/.venv/lib/python2.7/site-packages/flask/exthook.py", line 87, in load_module
    raise ImportError('No module named %s' % fullname)
ImportError: No module named flask.ext.restful.inputs
/Users/quazinafiulislam/Code/Python/restful/docs/api.rst:48: WARNING: autodoc: failed to import function u'boolean' from module u'flask.ext.restful.inputs'; the following exception was raised:
Traceback (most recent call last):
  File "/Users/quazinafiulislam/Code/Python/restful/.venv/lib/python2.7/site-packages/sphinx/ext/autodoc.py", line 335, in import_object
    __import__(self.modname)
  File "/Users/quazinafiulislam/Code/Python/restful/.venv/lib/python2.7/site-packages/flask/exthook.py", line 87, in load_module
    raise ImportError('No module named %s' % fullname)
ImportError: No module named flask.ext.restful.inputs
/Users/quazinafiulislam/Code/Python/restful/docs/api.rst:49: WARNING: autodoc: failed to import function u'rfc822' from module u'flask.ext.restful.inputs'; the following exception was raised:
Traceback (most recent call last):
  File "/Users/quazinafiulislam/Code/Python/restful/.venv/lib/python2.7/site-packages/sphinx/ext/autodoc.py", line 335, in import_object
    __import__(self.modname)
  File "/Users/quazinafiulislam/Code/Python/restful/.venv/lib/python2.7/site-packages/flask/exthook.py", line 87, in load_module
    raise ImportError('No module named %s' % fullname)
ImportError: No module named flask.ext.restful.inputs
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] testing
writing additional files... genindex search
copying static files... done
copying extra files... done
dumping search index... done
dumping object inventory... done
build succeeded, 10 warnings.

Build finished. The HTML pages are in _build/html.

score 0 · Answer 1 · answered Oct 11 '14 at 14:06

If you want to install a module from SCM (git, hg, etc checkout), and do it without much pain, then:

create a virtualenv
checkout the working copy to wherever you want (not necessarily in your project folder),
link the checked out module in development mode in the virtualenv with
```
python setup.py develop
```
if supported, or
```
python setup.py install
```
if it requires installation.

After these steps flask_restful should be importable as flask.ext.restful.

Generating documentation for flask extensions using sphinx

1 Answers1