How to determine whether code is running in a doctest?

Question

How can I make my (Python 2.7) code aware whether it is running in a doctest?

The scenario is as follows: I have a function that print()s some output to a file descriptor passed in as an argument, something like this:

from __future__ import print_function

def printing_func(inarg, file=sys.stdout):
    # (do some stuff...)
    print(result, file=file)

But when I try to use printing_func() in a doctest, the test fails; because of my specification of the keyword argument file when invoking print(), the output actually goes to sys.stdout rather than whatever default output redirection is set by the doctest module, and doctest never sees the output.

So how can I make printing_func() aware whether it is running inside a doctest, so that it knows not to pass the file keyword argument when calling print()?

Chris Johnson · Answer 1 · 2019-02-19T22:46:06.647

Niten's version of inside_doctest seems too broad. It isn't that unusual to redefine sys.stdout, either for logging or when testing in a framework other than doctest, so it would give false positives.

A narrower test looks like this:

import sys

def in_doctest():
    """
Determined by observation
    """
    if '_pytest.doctest' in sys.modules:
        return True
    ##
    if hasattr(sys.modules['__main__'], '_SpoofOut'):
        return True
    ##
    if sys.modules['__main__'].__dict__.get('__file__', '').endswith('/pytest'):
        return True
    ##
    return False


def test():
    """
    >>> print 'inside comments, running in doctest?', in_doctest()
    inside comments, running in doctest? True
    """
    print 'outside comments, running in doctest?', in_doctest()

if __name__ == '__main__':
    test()

in_doctest tests for the _SpoofOut class doctest uses to replace sys.stdout. There are other attributes of the doctest module that could be verified the same way. Not that you can prevent another module from reusing a name, but this name isn't common, so probably a decent test.

Put the above in test.py. Running it in non-doctest mode, python test.py yields:

outside comments, running in doctest? False

Running in doctest verbose mode, python -m doctest test.py -v yields:

Trying:
    print 'inside comments, running in doctest?', in_doctest()
Expecting:
    inside comments, running in doctest? True
ok

I agree with others' comments that making code aware of doctest is generally a bad idea. I've only done it in somewhat exotic circumstances -- when I needed to create test cases via a code generator because there were too many to efficiently craft manually. But if you need to do it, the above is a decent test for it.

score 1 · Accepted Answer · answered Nov 14 '11 at 01:06

1

I figured out the answer after reading doctest.py; posting here for posterity...

Doctest redirects standard output by assigning a new file descriptor to sys.stdout. The problem was that my function description was closing over the value of the original sys.stdout file descriptor prior to doctest's redefinition.

Instead, if I do the following:

def printing_func(inarg, file=None):
    # (do some stuff...)

    if file is None:
        file = sys.stdout

    print(result, file=file)

then printing_func() will capture the sys module rather than sys.stdout, and when it runs it will retrieve doctest's redefined stdout attribute from sys if running inside a test.

EDIT: This also yields an easy way to check whether we are running inside a doctest:

def inside_doctest(original_stdout=sys.stdout):
    return original_stdout != sys.stdout

answered Nov 14 '11 at 01:06

mshroyer

1,928
17
14

That is a much better plan than trying to figure out if you're inside a doctest. – Raymond Hettinger Nov 14 '11 at 01:11
I agree with Raymond; the very idea of adding code to your system to check whether it's being tested and make it behave differently is definitely not ideal for testing methodology! – Ben Nov 14 '11 at 01:30
I agree in general, but I don't think it's harmful if one only does so for the sake of supporting the testing infrastructure. – mshroyer Nov 14 '11 at 02:01
@Niten's version of `inside_doctest` is a bit too broad -- the chance of false positives is too high for my tastes. See my answer for a tighter solution. – Chris Johnson Apr 12 '14 at 11:46

score 0 · Answer 3 · answered Dec 14 '13 at 20:12

FWIW (and sorry to be late and redundant) many developers regard "if test" as an antipattern.

I.e. if your code under test does different things when it's being tested than when it is run "for real," you're asking for trouble. Even if you believe you are doing it for a good reason. Hence the comments above applauding your solution that doesn't do that. When I'm tempted to use the "if test" pattern I try to refactor things so it isn't needed.

score 0 · Answer 4 · answered Jan 16 '19 at 17:18

0

I just check if the module 'doctest' has been loaded.

def in_doctest():
    import sys
    return 'doctest' in sys.modules

answered Jan 16 '19 at 17:18

barjak

10,842
3
33
47

How to determine whether code is running in a doctest?

4 Answers4