Unit Testing With/Without hardward in the loop

Question

Background

I am working on a project that interfaces with various cameras and their unique APIs. All cameras are abstracted into one interface, and the camera-specific API calls are contained within their own implementing classes.

Problem Explanation

I am trying to test a factory method that parses a form of configuration, and returns a pointer to the newly created camera. Here is a high-level view of what that looks like:

// camerafactory.cpp 

Camera* make_camera(Config& config) {
    switch (config.type) {
        case CameraTypes.A : {
            int paramInt = config.paramInt;
            return new CameraA(paramInt);
        }
        case CameraTypes.B : {
            string paramString = config.paramString;
            return new CameraB(paramString);
        }
        case CameraTypes.C : {
            float param1 = config.param1;
            float param2 = config.param2;
            return new CameraC(param1, param2);
        }
        default: {
            throw std::invalid_argument("Invalid configuration!");
        }

    }
}

I want to test that these constructors are getting called correctly, however this would require the cameras to be physically connected to the computer to allow for the camera API's within the camera constructors to work. This does make sense for some testing situations, i.e. demonstrating hardware in the loop, however in my specific case, it is unrealistic to have all types of cameras connected at a single time.

I would like to test that these constructors are at least getting called with the correct parameters, demonstrating that the parsing is working correctly. I have considered abstracting out the parsing and testing that separately, but technically there is still the constructor call that is not being tested. I was also always told that adding functionality for the sake of testing is usually something that should be avoided, so that is another thing I am considering with my solution. It would also be interesting to find a solution that allows easy compile-time switching between mocks/concrete implementations to test hardware in the loop.

Versions

Note, I am using C++ 17, CMake, and the latest version of gtest.

Solutions being considered

Using macros to change the constructor calls to Mock constructor calls.

At the top of camerafactory.cpp, adding something like:

// camerafactory.cpp

#ifndef PRODUCTION
#include "MockCamera.hpp"
#define CameraA MockCameraA
#define CameraB MockCameraB
#define CameraC MockCameraC
#endif

Where MockCameraN are defined in a separate hpp/cpp file with identical constructors, and also implement the same Camera Interface, which I can then use to test correct construction.

This would allow me to test that the constructors are getting called correctly, but does feel like it goes against a cleaner solution that could be done with a language like Java.

Using GMock

I have not looked too deeply into it yet, but GMock seems to be promising: http://google.github.io/googletest/gmock_cook_book.html

Answer 1

My advice with and without gmock is to use virtual methods that can be overridden in test code.

First, convert your factory funciton, make_camera, into its own class with virtual methods for things you want to validate. For brevity, I'm using an inline approach. In production code, you would probably want to separate this out into its own .h file for the class declaration and .cpp file for the implementation.

class CameraFactory
{
public:

    virtual Camera* make_camera_a(int param) {
        return new CameraA(param);
    }

    virtual Camera* make_camera_b(const string& param) {
        return new CameraB(param);
    }

    virtual Camera* make_camera_c(double param1, param2) {
        return new CameraC(param1, param2);
    }

    virtual Camera* make_camera(const Config& config) {
        switch (config.type) {
            case CameraTypes::A: {
                return make_camera_a(config.intParam);
            }
            case CameraTypes::B: {
                return make_camera_b(config.strParam);
            }
            case CameraTypes::C: {
                return make_camera_c(config.param1, config.param2);
            }
            default: {
                throw std::invalid_argument("Invalid configuration!");
            }
        }
    }
};

In your product code, where you would ordinarily invoke make_camera, you can just use a CameraFactory off the stack:

Replace lines like this in your product code:

Camera* cam = make_camera(config); // old code

With this:

CameraFactory factory;
Camera* cam = factory.make_camera(config);

Testing without gmock:

Old way of testing this would be for your unit test to override the individual make functions in a class that partially mocks out the virtual methods you want to validate getting called correctly.

class MockCamera : public Camera
{
};

class MockCameraFactory : public CameraFactory
{
    virtual Camera* make_camera_a(int param) {
        _camera_a_created = true;
        _intParm = param;
        return new MockCamera();
    }

    virtual Camera* make_camera_b(const string& param) {
        _camera_b_created = true;
        _strParam = param;
        return new MockCamera();
    }

    virtual Camera* make_camera_c(double param1, param2) {
        _camera_c_created = true;
        _param1 = param1;
        _param2 = param2;
        return new MockCamera();
    }

    bool _camera_a_created;
    bool _camera_b_created;
    bool _camera_c_created;
    int _intParam;
    string _strParam;
    double _param1;
    double _param2;

    CameraFactoryTest() {
        resetState();
    }
    void resetState() {
        _camera_a_created = false;
        _camera_b_created = false;
        _camera_c_created = false;
        _intParam = 0;
        _strParam = "";
        _param1 = 0;
        _param2 = 0;
    }
};

Notice that CameraFactoryTest doesn't override make_camera, but it does inherit from it.

Then your unit test code can validate that the right parameters are getting passed to your constructors. I'm just guessing your ut framework has some validation macro called UNIT_TEST_ASSERT. I'm sure you have something equivalent.

void test_make_camera()
{
    MockCameraFactory factory;
    Config config;

    // test camera A
    config.type=CameraConfig::A;
    config.intParam = 42;
    factory.make_camera(config);
    UNIT_TEST_ASSERT(factory._camera_a_created == true);
    UNIT_TEST_ASSERT(factory._camera_b_created == false);
    UNIT_TEST_ASSERT(factory._intParam == 42);

    // test camera B
    factory.resetState();
    config = {};
    config.type=CameraConfig::B;
    config.strPara = "USB1";
    factory.make_camera(config);
    UNIT_TEST_ASSERT(factory._camera_b_created == true);
    UNIT_TEST_ASSERT(factory._strParam == "USB1");

    ...

}

With Google Test (GMock)

It's even easier with gmock and enables your mock class to be more usable in other tests. Instead of explicitly coding up a MockCameraFactory class, you use the GMock way.

Start by declaring a mock class that inherits from your actual class.

class MockCameraFactory : public CameraFactory
{
public:
    MOCK_METHOD1(make_camera_a, Camera*(int param));
    MOCK_METHOD1(make_camera_b, Camera*(const string& param));
    MOCK_METHOD2(make_camera_c, Camera*(const string& param));
};

Then your test code can do something like this:

class test_make_camera()
{
    MockCameraFactoryTest factory;
    MockCamera cameramock;

    EXPECT_CALL(factory, make_camera_a(42)).WillOnce(Return(&cameramock));

    Config config = {};
    config.type = CameraType::A;
    config.intParam = 42;

    factory.make_camera(config);

    Mock::VerifyAndClearExpectations(&factory);

}

I'm skimming over the details of how to setup Google Test such that it asserts through your own unit test framework. But you get the idea.

Also of note - I originally declared make_camera to be virtual in the CameraFactory class. It doesn't actually have to be virtual since your tests are only validating that the make_camera_x methods are getting called correctly.