How to correctly mock a gcp client library call in python

Question

How do I correctly write a unit test for a function that uses the GCP secret manager client library. I've been reading up on unit testing and mocking but I just can't seem to grasp what's going wrong here. I've never really written unit tests other than very basic ones, or done mocking either. I have the following get_secret function in a file main.py that returns a string.

from google.cloud import secretmanager

def get_secret(project_id,secret_name) -> str:
    """
    Get secret from gcp secrets manager
    """

    client = secretmanager.SecretManagerServiceClient()
    request = {"name": f"projects/{project_id}/secrets/{secret_name}/versions/latest"}

    response = client.access_secret_version(request)
    secret_string = response.payload.data.decode("UTF-8")

    return secret_string

I have the following test_main.py file where I try to mock the secretmanager.

import pytest
from unittest.mock import patch

from main import get_secret

@pytest.fixture()
def secret_string():
    return 'super_secret_token'

@patch("main.secretmanager") # mock secretmanager from main.py
def test_get_secret(secretmanager,secret_string):

    secretmanager.SecretManagerServiceClient().access_secret_version().return_value = secret_string

    secret_string = get_secret('project_id','secret_name')

    assert secret_string == 'super_secret_token'

When I run pytest it test fails with AssertionError: assert <MagicMock name='secretmanager.SecretManagerServiceClient().access_secret_version().payload.data.decode()' id='4409262192'> == 'super_secret_token'

I have an idea why but I'm not entirely sure. I assume it's to do with access_secret_version() returning an object of type google.cloud.secretmanager_v1.types.service.AccessSecretVersionResponse which has a payload object of type google.cloud.secretmanager_v1.types.SecretPayload which is a data object of type bytes

Any help on how to do this correctly would be appreciated.

Instead of trying to stub the calls to Secret Manager, why not stub the call to `get_secret`? — sethvargo, Aug 16 '21 at 16:14
I think it's better to create your own class `SecretsManager` that calls GCP. Then use that throughout your code. Since it's pretty small it should be possible to mock it very easily and it will work throughout your codebase — Pithikos, Apr 06 '23 at 06:53
@Pithikos Seems like a better approach. I'll give it a go the next time. — never_odd_or_even, Apr 08 '23 at 00:46

never_odd_or_even · Accepted Answer · 2021-08-31T01:32:49.157

@patch("main.secretmanager") attempts to patch the secretmanager which is a module but I need to be patching secretmanager.SecretManagerServiceClient which is a class, I had tried this but it was giving me errors due to the incorrect syntax I was using for return_value. It's now working with.

@patch("main.secretmanager.SecretManagerServiceClient")
def test_get_secret(self, mock_smc):

  mock_smc.return_value.access_secret_version.return_value.payload.data = b'super_secret_token'
  secret_string = get_secret('project_id', 'secret_name')

  assert secret_string == 'super_secret_token'

You could also patch access_secret_version() but when running the test it will attempt to authenticate with gcp when client = secretmanager.SecretManagerServiceClient() is called and if there are no valid credentials it will fail. It's probably best not to authenticate with external service when running unit tests.

from unittest.mock import patch
from main import get_secret

@patch("main.secretmanager.SecretManagerServiceClient.access_secret_version")
def test_get_secret(secretmanager):

    secretmanager.return_value.payload.data = b'super_secret_token'
    secret_string = get_secret('project_id', 'secret_name')

    assert secret_string == 'super_secret_token'

I have a question. What is mock_smc? Can you please paste the fixture here? — Somil Aseeja, Mar 11 '22 at 15:47
`mock_smc` is the mocked version of the `SecretManagerServiceClient` class we created it with `@patch`. When using the `@patch` decorator you name the mock object as an argument in the decorated function. See the example of two patched classes using the patch decorator [here](https://docs.python.org/3/library/unittest.mock.html#quick-guide). Two classes are mocked using `@patch` and passed as arguments to the function, named `MockClass1` and `MockClass2`, they must be named in the order they were passed in, from the bottom up. Hope that makes sense. — never_odd_or_even, Mar 13 '22 at 23:45

How to correctly mock a gcp client library call in python

1 Answers1