Python docstrings templated

Question

Why doesn't dynamically formatting docstrings work? Is there an acceptable workaround for doing this at function definition time?

>>> DEFAULT_BAR = "moe's tavern"
>>> def foo(bar=DEFAULT_BAR):
...     """
...     hello this is the docstring
...     
...     Args:
...       bar (str)    the bar argument (default: {})
...     """.format(DEFAULT_BAR)
... 
>>> foo.__doc__
>>> foo.__doc__ is None
True

I tried with old-skool style %s formatting and that didn't work either.

Because it is the parser that takes the first string literal in the body. No code is executed for that. — Martijn Pieters, Apr 18 '16 at 22:08
Yes, I assumed it was the parser's job, but I've expected SyntaxError or something here rather than silent failure — wim, Apr 18 '16 at 22:09
It's valid syntax. You're allowed to put expressions as standalone statements (that's what a function call is), they just might not do anything. — Jeremy, Apr 18 '16 at 22:10
If you want to do this, you'll have to set `__doc__` yourself. A helper decorator to set `__doc__` might make things cleaner. — user2357112, Apr 18 '16 at 22:22

Martijn Pieters · Answer 1 · 2016-04-19T07:05:30.990

Your string requires the function to be called, but function bodies are not executed when a function is created.

A proper docstring is not executed, it is simply taken from the parsed sourcecode and attached to the function object, no code is ever executed for this. Python stores the docstring as the first constant value in a code object:

>>> def f():
...     """docstring"""
...     pass
...
>>> f.__code__.co_consts
('docstring', None)

where the code object was passed to the function type when constructing a new function (see the PyFunction_New() function).

See the Function definitions reference documentation:

The function definition does not execute the function body; this gets executed only when the function is called. [3]

[...]

[3] A string literal appearing as the first statement in the function body is transformed into the function’s __doc__ attribute and therefore the function’s docstring.

Your definition is otherwise valid; there is just no stand-alone string literal at the top of the function body. Your string literal is instead part of the function itself, and is only executed when the function is called (and the result discarded as you don't store that).

Note that the __doc__ attribute on a function object is writable; you can always apply variables after creating the function:

>>> DEFAULT_BAR = "moe's tavern"
>>> def foo(bar=DEFAULT_BAR):
...     """
...     hello this is the docstring
...
...     Args:
...       bar (str)    the bar argument (default: {})
...     """
...
>>> foo.__doc__ = foo.__doc__.format(DEFAULT_BAR)
>>> print(foo.__doc__)

    hello this is the docstring

    Args:
      bar (str)    the bar argument (default: moe's tavern)

You could do that in a decorator with the help of functionobject.__globals__ and inspect.getargspec() perhaps, but then do use named slots in the template so you can apply everything as a dictionary and have the docstring choose what to interpolate:

from inspect import getargspec

def docstringtemplate(f):
    """Treat the docstring as a template, with access to globals and defaults"""
    spec = getargspec(f)
    defaults = {} if not spec.defaults else dict(zip(spec.args[-len(spec.defaults):], spec.defaults))
    f.__doc__ = f.__doc__ and f.__doc__.format(**dict(f.__globals__, **defaults))
    return f

Demo:

>>> @docstringtemplate
... def foo(bar=DEFAULT_BAR):
...     """
...     hello this is the docstring
...
...     Args:
...       bar (str)    the bar argument (default: {bar!r}, or {DEFAULT_BAR!r})
...
...     """
...
>>> print(foo.__doc__)

    hello this is the docstring

    Args:
      bar (str)    the bar argument (default: "moe's tavern", or "moe's tavern")

Function keyword arguments override globals, as they would in the function.

I don't believe saying the string literal is executed is accurate. If I make the DEFAULT_BAR an object which raises exception when the `__str__` method is called, that does not happen. — wim, Apr 18 '16 at 22:17
@wim He meant it would be executed when the function is called (which I just verified for myself). — Jeremy, Apr 18 '16 at 22:39
@wim: yes, I meant when the function is called; added that to the answer. — Martijn Pieters, Apr 18 '16 at 23:01

score 2 · Accepted Answer · answered Apr 18 '16 at 23:58

Try something like this (props to @user2357112 for the suggestion):

#!python3

def FORMAT_DOC(f):
    """Decorator to format docstring of a function. Supplies 
    `defaults` dictionary, positional values, and argname:value 
    pairs to format - use {defaults[a]} or {a} or {0} to access
    the 0th default value, etc.
    """
    defaults = f.__defaults__
    docs = f.__doc__

    if docs and defaults:
        nargs = f.__code__.co_argcount
        argnames = f.__code__.co_varnames[nargs-len(defaults):nargs]
        argdefaults = dict(zip(argnames, defaults))
        f.__doc__ = docs.format(defaults=argdefaults, *defaults, **argdefaults)

    return f

@FORMAT_DOC
def f(a):
    pass

@FORMAT_DOC
def f(a,b,c=1,d=2):
    """Docstring

    By default, c = {} and d = {}
    """
    v=a+c
    w=b+d
    x=w/v
    return x+1

@FORMAT_DOC
def f(a=0, b="foo", c="bar"):
    """Docstring: a={0}, b={defaults[b]}, c={c}"""
    pass

Python docstrings templated

2 Answers2

Linked