Multiple calls to botocore.stub.Stubber causes exception

Question

I am using botocore.stub.Stubber to mock the kmsclient. The code I am using is

with botocore.stub.Stubber(s3) as stubber:
        with botocore.stub.Stubber(kms) as stubber2:
            stubber.add_response('copy_object', response, expectedParams)
            stubber.activate()
            stubber2.add_response('decrypt', response2, expectedParams2)
            stubber2.activate()
            handleCore(__makeValidEvent(), None, s3, kms)
            stubber.assert_no_pending_responses()
            stubber2.assert_no_pending_responses()

With the actual implementation the kmsclient call is happening twice which results in the following exception

params = {'CiphertextBlob': b"\x01\x02\x02\x00x#\xc1\xdbp6\xe1Y\x0fS\x15\x80<\x86\xb5\xb2\x86\x9f\xaf\xa2Z\x07\xfef\x8d\xb2\xd7...\t'\xe2\xb9\x10w0\x83\xcb\xe1\xcb`\xd1\xc2\x8c\xe4\x82Q/*\xb3]\xcfZ\xb9\xbd\x1c\x9c\x96(e\x94j\x1a\x91\xba\xaeO[>\x97"}

    def _assert_expected_call_order(self, model, params):
        if not self._queue:
            raise UnStubbedResponseError(
                operation_name=model.name,
                reason=(
>                       'Unexpected API Call: A call was made but no additional calls expected.'
                        'Either the API Call was not stubbed or it was called multiple times.'
                        )
            )
E           botocore.exceptions.UnStubbedResponseError: Error getting response stub for operation Decrypt: Unexpected API Call: A call was made but no additional calls expected.Either the API Call was not stubbed or it was called multiple times.

Can someone let me know how can could be used for multiple calls on the same object(kmsclient in this case)

I have found an alternate solution using moto – Kailas J C Jun 13 '18 at 10:50 — Kailas J C, Jun 13 '18 at 10:50
from moto import ( mock_kms, mock_s3, ) – Kailas J C Jun 13 '18 at 10:50 — Kailas J C, Jun 13 '18 at 10:50

score 4 · Answer 1 · answered Apr 09 '19 at 18:07

I solved this by adding multiple responses for the same method.

Example test code:

############## Test Code
client = botocore.session.get_session().create_client("cloudformation", region_name="us-east-1")
stubber = Stubber(client)

# Generate multiple responses to be returned by boto
responses = [
    {
        "StackId": "stack-a",
    },
    {
        "StackId": "stack-b",
    },
    {
        "StackId": "stack-c",
    },
]

# Add each response to stubber for the same method - "update_termination_protection"
for response in responses:
    stubber.add_response(
        "update_termination_protection",
        response,
    )
stubber.activate()
actual = method_to_test(client, data)
stubber.deactivate()
assert actual == True

Example method to test:

############## Real method
def method_to_test(self, client, data):
    for item in data:
        client.update_termination_protection(
           EnableTerminationProtection=True,
           StackName=item,
        )
    return True

This works and throws no exceptions.

score 1 · Answer 2 · answered Apr 29 '20 at 17:29

I had this same problem but found I could fix it by calling deactivate() before I set up the stub.

So, your example would be:

with botocore.stub.Stubber(s3) as stubber:
        with botocore.stub.Stubber(kms) as stubber2:
            stubber.deactivate()
            stubber.add_response('copy_object', response, expectedParams)
            stubber.activate()
            stubber2.deactivate()
            stubber2.add_response('decrypt', response2, expectedParams2)
            stubber2.activate()
            handleCore(__makeValidEvent(), None, s3, kms)
            stubber.assert_no_pending_responses()
            stubber2.assert_no_pending_responses()

I know your answer is copying the layout of op's example code for simplicity, but the calls to .deactivate() and .activate() can be avoided by moving the .add_response() calls before the with statements. — AlwaysLearning, Jan 20 '21 at 02:24

score 0 · Answer 3 · answered Jan 20 '21 at 02:33

As per the Stubber.add_response() documentation:

Adds a service response to the response queue.

So if you expect multiple calls to a stubbed method then your test should invoke add_response() multiple times, each with the response and expected params in the order they should be consumed.

So your test code with two calls to kms.decrypt would be arranged like the following:

stubber.add_response('copy_object', response, expectedParams)
stubber2.add_response('decrypt', response2, expectedParams2)
stubber2.add_response('decrypt', response2, expectedParams2)
with botocore.stub.Stubber(s3) as stubber:
        with botocore.stub.Stubber(kms) as stubber2:
            handleCore(__makeValidEvent(), None, s3, kms)
            stubber.assert_no_pending_responses()
            stubber2.assert_no_pending_responses()

You may wind up using response2, expectedParams2 and response3, expectedParams3 though if the first and second calls to kms.decrypt are doing different things.

How is python able resolve the name `stubber` outside of the `with` statement? — Jesusaur, Sep 01 '22 at 22:21

Multiple calls to botocore.stub.Stubber causes exception

3 Answers3