0

I am developing an application that it's meant to be run on linux, and relies on pty and termios modules (I don't think it's important, but I'm writing some DAGs on Airflow). I use a windows workstation. I would like to run the unit tests on my local machine, and I'm trying to use unittest.mock.patch to prevent the import of the (non existent on my OS) modules.

Still, I have problems, and I don't know how to setup the import to avoid errors.

a minimal example is

lib.py

import termios
class C: ...

foo.py

from lib import C
def bar(): ...

test_foo.py

import foo

def test_foo_bar():
    assert foo.bar() == ...

what "magic" should I write in test_foo to avoid the error

>pytest test_foo.py
============================================================================================ test session starts =============================================================================================
platform win32 -- Python 3.7.9, pytest-7.0.1, pluggy-1.0.0
rootdir: C:\Users\vito.detullio\Desktop\workspace-srm-cognitive\be-airflow-dags, configfile: pytest.ini
plugins: anyio-3.4.0, cov-3.0.0
collected 0 items / 1 error                                                                                                                                                                                   

=================================================================================================== ERRORS ===================================================================================================
_____________________________________________________________________________________ ERROR collecting delme/test_foo.py _____________________________________________________________________________________
ImportError while importing test module 'C:\Users\vito.detullio\Desktop\workspace-srm-cognitive\be-airflow-dags\delme\test_foo.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
c:\Program Files\Python37\lib\importlib\__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
test_foo.py:1: in <module>
    import foo
foo.py:1: in <module>
    from lib import C
lib.py:1: in <module>
    import termios
E   ModuleNotFoundError: No module named 'termios'
========================================================================================== short test summary info ===========================================================================================
ERROR test_foo.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================================================== 1 error in 0.14s ==============================================================================================
Vito De Tullio
  • 2,332
  • 3
  • 33
  • 52

2 Answers2

0

You first have to make sure that you don't import the non-existing module via a top-level import. If you import it only locally, you can add some placeholder module to the module cache before doing the import:

import sys
from unittest import mock


def test_foo_bar():
    if "termios" not in sys.modules:
        sys.modules["termios"] = mock.MagicMock()
    import foo
    assert foo.bar() == ...

This of course assumes that you don't actually need the module in your test.

Note that you also cannot patch bar or something else that lives in that module via a patch decorator`, as that would also import the module before you have a chance to patch it, even if it does not directly reference it. You have to do the patching inside the test instead.

MrBean Bremen
  • 14,916
  • 3
  • 26
  • 46
0

I had the same issues with airflow recently. Inspired by this answer, I resolved mocking all airflow-correlated modules. The idea is to edit the module map using the sys.modules dictionary from sys module contained in Python Standard Library. For example, if you need to mock only task from decorators.airflow module (that is not working on Windows), you can use the following code:

airflow_module = type(sys)("airflow")
airflow_module.decorators = type(sys)("decorators")
airflow_module.decorators.task = MagicMock()
sys.modules["airflow"] = airflow_module
sys.modules["airflow.decorators"] = airflow_module.decorators

You can mock every airflow submodule using this method (that also works for submodules). For example, in my case, I add to conftest.py the following code to mock task, get_current_context, Variable and TriggerRule from different airflow submodules:

if platform.system() == 'Windows':
    airflow_module = type(sys)("airflow")
    airflow_module.decorators = type(sys)("decorators")
    airflow_module.operators = type(sys)("operators")
    airflow_module.operators.python = type(sys)("python")
    airflow_module.models = type(sys)("models")
    airflow_module.models.variable = type(sys)("variable")
    airflow_module.utils = type(sys)("utils")
    airflow_module.utils.trigger_rule = type(sys)("trigger_rule")

    airflow_module.decorators.task = MagicMock()
    airflow_module.operators.python.get_current_context = MagicMock()
    airflow_module.models.variable.Variable = MagicMock()
    airflow_module.utils.trigger_rule.TriggerRule = MagicMock()

    sys.modules["airflow"] = airflow_module
    sys.modules["airflow.decorators"] = airflow_module.decorators
    sys.modules["airflow.operators"] = airflow_module.operators
    sys.modules["airflow.operators.python"] = airflow_module.operators.python
    sys.modules["airflow.models"] = airflow_module.models
    sys.modules["airflow.models.variable"] = airflow_module.models.variable
    sys.modules["airflow.utils"] = airflow_module.utils
    sys.modules["airflow.utils.trigger_rule"] = airflow_module.utils.trigger_rule
FabioL
  • 932
  • 6
  • 22