How to document structures in python?

Question

If an argument to a function is expected to be a certain (or equivalent) structure, built using python's list, tuple and dict, how and where should it be documented?

An example documentation:

def foo(bar):
    """
    Args:
        bar: 2-tuple, ([(<mapping>,<number>), ...], <string>)
    """
    pass

A bit cumbersome; some problems:

• the structure is hard to read
• hard to indicate semantical meaning of each element of the structure
• how to indicate not-fix length
• should it only be documented once, or everywhere
• edit: how to clearly show that duck-typing is ok for an element (ie. "dict" vs. "mapping-like")

edit: The example is not for trying to enforce types, it's trying to document a structure. For the point, duck typing is ok.

Answer 1

I ran into this problem a while back in my work. Based on my personal research (ie a decently thorough Google search) there is no standard for documenting complex, nested function arguments. I ended up inventing my approach, which uses <> to denote elements of the data structure, and each element gets its own header and paragraph in the docs. For example:

<options>
---------

A mapping:

  {'years':<years>,
   'states':<states>}

This is the root level of the `options` argument.

<years>
-------

A sequence of integers, each of which is a year to be included in the analysis.


<states>
--------

A sequence of strings, each of which is a state to be included in the analysis.

This documentation could be included in the doc strings for the function, but in my case I opted to break it out into a separate document. The approach can be extended to more complicated structures. See for example my docs here.

Answer 2

My simple answer to your question is to quote the Zen of Python: "Explicit is better than implicit". In terms of your question, this means write out in full exactly what's needed, even if it takes an entire paragraph to document a single argument. Just read the python docs to see examples. In your specific case, you can indicate duck-typing by referring to a sequence rather than a tuple (see http://docs.python.org/2/library/collections.html#collections-abstract-base-classes).

Answer 3

If you are using Python 3, you might be interested in "function annotations". They're not enforced, and completely optional but seem to do what you want. An annotation can be any Python expression, and they are included in the function header, next to the arguments. For example (sorry it's a bit nonsensical):

def fn(name: str, age: int, hobbies: "something else goes here") -> max(2, 9):
  # la la la
  return 9

There are more details (including possible use cases and examples) here: PEP 3107

Answer 4

You can use a decorator to force types and also on the way document it:

def require(arg_name, *allowed_types):
    '''
    example:
    @require("x", int, float)
    @require("y", float)
    def foo(x, y):
        return x+y
    '''
    def make_wrapper(f):
        if hasattr(f, "wrapped_args"):
            wrapped_args = getattr(f, "wrapped_args")
        else:
            code = f.func_code
            wrapped_args = list(code.co_varnames[:code.co_argcount])

        try:
            arg_index = wrapped_args.index(arg_name)
        except ValueError:
            raise NameError, arg_name
        def wrapper(*args, **kwargs):
            if len(args) > arg_index:
                arg = args[arg_index]
                if not isinstance(arg, allowed_types):
                    type_list = " or ".join(str(allowed_type) for allowed_type in allowed_types)
                    raise Exception, "Expected '%s' to be %s; was %s." % (arg_name, type_list, type(arg))
            else:
                if arg_name in kwargs:
                    arg = kwargs[arg_name]
                    if not isinstance(arg, allowed_types):
                        type_list = " or ".join(str(allowed_type) for allowed_type in allowed_types)
                        raise Exception, "Expected '%s' to be %s; was %s." % (arg_name, type_list, type(arg))

            return f(*args, **kwargs)

        wrapper.wrapped_args = wrapped_args
        return wrapper

    return make_wrapper        

Answer 5

The most complete docstring specification I've seen is the NumPy Docstring Style Guide, but I agree it doesn't give the amount of detail to cover your question. As such I might suggest that a standard may not exist to answer your (and my own) question at the moment.

Answer 6

I think my least favorite function , ever, had really good inline docs.

Check out the validate docs here:

Pylons Decorators - Validate - GitHub

def validate(schema=None, validators=None, form=None, variable_decode=False,
         dict_char='.', list_char='-', post_only=True, state=None,
         on_get=False, **htmlfill_kwargs):
"""Validate input either for a FormEncode schema, or individual
validators

Given a form schema or dict of validators, validate will attempt to
validate the schema or validator list.

If validation was successful, the valid result dict will be saved
as ``self.form_result``. Otherwise, the action will be re-run as if
it was a GET, and the output will be filled by FormEncode's
htmlfill to fill in the form field errors.

``schema``
    Refers to a FormEncode Schema object to use during validation.
``form``
    Method used to display the form, which will be used to get the
    HTML representation of the form for error filling.
``variable_decode``
    Boolean to indicate whether FormEncode's variable decode
    function should be run on the form input before validation.
``dict_char``
    Passed through to FormEncode. Toggles the form field naming
    scheme used to determine what is used to represent a dict. This
    option is only applicable when used with variable_decode=True.
``list_char``
    Passed through to FormEncode. Toggles the form field naming
    scheme used to determine what is used to represent a list. This
    option is only applicable when used with variable_decode=True.
``post_only``
    Boolean that indicates whether or not GET (query) variables
    should be included during validation.

    .. warning::
        ``post_only`` applies to *where* the arguments to be
        validated come from. It does *not* restrict the form to
        only working with post, merely only checking POST vars.
``state``
    Passed through to FormEncode for use in validators that utilize
    a state object.
``on_get``
    Whether to validate on GET requests. By default only POST
    requests are validated.

Example::

    class SomeController(BaseController):

        def create(self, id):
            return render('/myform.mako')

        @validate(schema=model.forms.myshema(), form='create')
        def update(self, id):
            # Do something with self.form_result
            pass

"""