Include raw tab literal character in doctest

Question

I can't figure out how to avoid this doctest error:

Failed example:
    print(test())
Expected:
        output
    <BLANKLINE>
Got:
        output
    <BLANKLINE>

For this code

def test():
    r'''Produce string according to specification.

    >>> print(test())
        output
    <BLANKLINE>
    '''
    return '\toutput\n'

I have put a tab literal into the source code, line 5 in front of output.

It looks like doctest (or python docstrings?) ignores that tab literal and converts it to four spaces.

The so-called "Expected" value is literally not what my source specifies it to be.

What's a solution for this?

I don't want to replace the print statement with

>>> test()
'\toutput\n'

Because a huge part of why I usually like doctests is because they demonstrate examples, and the most important part of this function I am writing is the shape of the output.

Are you sure that it's not your editor/IDE replacing tabs with spaces? — snwflk, Oct 17 '20 at 23:37
It's worth noting that Stack Overflow converts tabs to four spaces. That might be what @snwflk is talking about. — wjandrea, Oct 18 '20 at 01:30
@wjandrea oh yes, I hadn't noticed that that's a shame. On my computer though, it's definitely a tab character. — theonlygusti, Oct 18 '20 at 08:32

score 5 · Accepted Answer · edited Oct 18 '20 at 15:50

The tabs in the docstring get expanded to 8 spaces, but the tabs in the output are not expanded.

From the doctest documentation (emphasis added):

All hard tab characters are expanded to spaces, using 8-column tab stops. Tabs in output generated by the tested code are not modified. Because any hard tabs in the sample output are expanded, this means that if the code output includes hard tabs, the only way the doctest can pass is if the NORMALIZE_WHITESPACE option or directive is in effect. Alternatively, the test can be rewritten to capture the output and compare it to an expected value as part of the test. This handling of tabs in the source was arrived at through trial and error, and has proven to be the least error prone way of handling them. It is possible to use a different algorithm for handling tabs by writing a custom DocTestParser class.

Unfortunately, all examples of directive use in the doctest docs have been mangled by Sphinx. Here's how you would use NORMALIZE_WHITESPACE:

def test():
    r'''Produce string according to specification.

    >>> print(test()) # doctest: +NORMALIZE_WHITESPACE
        output
    <BLANKLINE>
    '''
    return '\toutput\n'

Note that this treats all runs of whitespace as equal, rather than disabling the tab processing. Disabling the tab processing is absurdly cumbersome, and it's impossible through the usual doctest interface - you would need to subclass DocTestParser, manually reimplement doctest parsing, then construct instances of DocTestFinder, DocTestRunner, and your DocTestParser subclass and call their methods manually.

This is quite a shame. The offhand "[this] has proven to be the least error prone way of handling them" is quite frustrating too. In what cases is handling tabs in the user-expected way going to cause errors? — theonlygusti, Oct 18 '20 at 08:56

Include raw tab literal character in doctest

1 Answers1