How to write unit tests with files which directly or indirectly `import greengrasssdk`

Question

Whenever a file imports import greengrasssdk the unit tests fail, because the module greengrass_commondoesn't exists on my local machine and I cannot install it via pip.

I am executing the tests with PyCharm. The Greengrass lambda, I try to test, doesn't execute locally because of the same dependency problem (same exception). But as soon as the lambda is pushed to greengrass it works fine.

Here is the exception:

    import greengrasssdk
  File "C:\Python27\lib\site-packages\greengrasssdk\__init__.py", line 6, 
in <module>
    from .Lambda import StreamingBody
  File "C:\Python27\lib\site-packages\greengrasssdk\Lambda.py", line 10, in 
<module>
     from greengrass_common.function_arn_fields import FunctionArnFields
ImportError: No module named greengrass_common.function_arn_fields

A simplified code example is this:

import greengrasssdk
import logging

greengrass_iot_client = greengrasssdk.client('iot-data')
logger = logging.getLogger('logger')

def handler(event, context):
    logger.info('Event handler invoked with event: ' + str(event))

I get the following error message on the test (The test is excluded in a test folder, but no other dependency issues have been shown yet - I write this because some developers put their tests into the python code file. I heard that that tests outside of the source code file could result in import issues. Though this case is different since it happens also in the original code file.)

import unittest
import mock
import function

class SimpleTest(unittest.TestCase):

# NONE OF THE THREE PATCH WORK  Not in combination nor single
@mock.patch('greengrass_common')
@mock.patch('greengrass_common.function_arn_fields')
@mock.patch('greengrasssdk')
def test_that(self):
    pass

The test case is empty for simplification.

I expect the greengrass_common code to be existing outside the Greengrass code for me to write unit tests.

I come from the java world, but spoke to a few python developers. We didn't really find a solution. (Except try catching the import in the production code) , but that seems like the first step into bad software quality in the whole project.

I am very grateful for ideas/solutions/approaches and guidance.

Thank you very much :).

score 0 · Answer 1 · answered Feb 12 '19 at 20:42

You can import greengrasssdk conditionally if you detect that you're running on Greengrass and use boto3 if you're not. I wrote a quick tests that is working for me here that looks like this:

import socket
host = socket.gethostname()

client = None

# Create an IoT data client with Greengrass SDK on Greengrass, boto3 locally
if host == 'sandbox':
  import greengrasssdk
  client = greengrasssdk.client('iot-data')
else:
  import boto3
  client = boto3.client('iot-data')

Essentially check if the hostname is reported to be 'sandbox' and then use the Greengrass SDK, otherwise use boto3.

score 0 · Answer 2 · answered May 10 '19 at 12:16

Had this problem as well and use the patcher in my conftest.py file to patch over the Greengrass SDK e.g.

conftest.py

import pytest
from mock import MagicMock, patch

MockGreengrassSdk = MagicMock()

modules = {
    "greengrasssdk": MockGreengrassSdk
}
patcher = patch.dict("sys.modules", modules)
patcher.start()

Once you have defined MockGreengrassSdk as a MagicMock you can then mock out any method you want on it. The only thing I am not sure on is how you would change that per test if you wanted different results from the SDK per test.

The way I have got around that is to create a facade over the greengrasssdk which I can then mock out the methods on. In this case that facade must import greengrasssdk which is where the code above will stop your tests from failing.

In summary:

Create a facade over the greengrasssdk methods you want to use.
In the facade import the greengrasssdk.
Use the patcher code in a conftest.py file (as above) to patch over the Greengrass SDK.

Hope that helps.

score 0 · Answer 3 · answered Jun 05 '19 at 08:31

I just ran into the same problem and for me it worked by surrounding the import with a try except statement.

try:
    import greengrasssdk
    client = greengrasssdk.client('iot-data')
except Exception as e:
    import boto3
    client = boto3.client('iot-data')

How to write unit tests with files which directly or indirectly `import greengrasssdk`

3 Answers3