How does QTEST_MAIN set everything up automatically to run tests?

Question

I'm working on an existing project written in C++, the entry point for the application is:

QTEST_MAIN(className)

From the documentation I've read, this will create a standard C main() function, but it isn't at all clear on how the application tests are called or what the call order is or how its set-up.

Looking at the class in the project I have, there is no class constructor; the class itself is derived from QObject. It has 23 private slots: one of these is called "initTestCase"; the others are various tests all ending in "Test".

The slot "InitTestCase" contains a single call to setup logging filter rules and that is all. When the project is compiled and run, it executes tests, but I cannot see how or where the order comes from.

What is the macro QTEST_MAIN actually doing in my program, how are the slots being set up, and how does it know which tests to execute?

See https://github.com/qt/qtbase/blob/dev/src/testlib/qtest.h#L506 — eyllanesc, Jan 30 '20 at 08:48
@eyllanesc, thank you, but that still doesn't really explain what connects the slots and schedules the tests, there is no class constructor so where is the logic? — SPlatten, Jan 30 '20 at 08:50
I guess you have no information about the order because "unit tests", by definition, are not supposed to depend on each others. — Fareanor, Jan 30 '20 at 08:53
If you realize that macro has as its ultimate goal to call "QTest::qExec", there the magic happens. I put the link for you to start the search and understand how it works internally (although it is not necessary to understand it to implement a test) — eyllanesc, Jan 30 '20 at 08:53
@Fareanor, thank you, using the Qt Creator "Find/Replace -> Advanced Find -> Project ''myproject'" I've search for all instances of the slots, there is no connection code in the project and the slots are not mentioned anywhere except in the class CPP and Header files, so again what sets these up for use? — SPlatten, Jan 30 '20 at 08:55
@eyllanesc, still looking at it, but its a can of worms and poorly documented. — SPlatten, Jan 30 '20 at 08:56
@SPlatten I recommend you check the source code since testlib is a small module of more or less 30 classes so that you understand how it is actually implemented. — eyllanesc, Jan 30 '20 at 08:58
I emphasize, you do not need to know how testlib works internally (if you saw how qtdeclarative is implemented, you would say that qtestlib is a unicorn), so Qt does not document it because it tries to prevent developers from wasting time on it, and many things can change without notice. Everything you need for the tests are in the docs and the examples. — eyllanesc, Jan 30 '20 at 09:00
@eyllanesc, it seems that the order of calling is determined by the order the slots are defined in the class, however I just tried moving my slot to before the "initTestCase" slot and the "moc" generated cpp still calls "initTestCase" first then the slot I added. — SPlatten, Jan 30 '20 at 09:06
Is that initTestCase is a special method that will be executed before all other functions as indicated by [the docs](https://doc.qt.io/qt-5/qtest-overview.html#creating-a-test): *initTestCase() will be called before the first test function is executed.*. You realize that to know that it is not necessary to review the source code but that Qt clearly indicates it, consider everything that is indicated in the docs of Qt as a postulate that you do not have to discuss but only accept, and if what is not fulfilled is in the docs then it is probably a bug. — eyllanesc, Jan 30 '20 at 09:08

score 1 · Accepted Answer · edited Mar 30 '20 at 18:33

QTEST_MAIN forwards to an QTEST_MAIN_IMPL:

#define QTEST_MAIN(TestObject) \
int main(int argc, char *argv[]) \
{ \
    QTEST_MAIN_IMPL(TestObject) \
}

This QTEST_MAIN_IMPL is different depending on what QApplication you need (Widgets, Gui or Core). For Widgets, it looks like this:

#define QTEST_MAIN_IMPL(TestObject) \
    TESTLIB_SELFCOVERAGE_START(#TestObject) \
    QT_PREPEND_NAMESPACE(QTest::Internal::callInitMain)<TestObject>(); \
    QApplication app(argc, argv); \
    app.setAttribute(Qt::AA_Use96Dpi, true); \
    QTEST_DISABLE_KEYPAD_NAVIGATION \
    TestObject tc; \
    QTEST_SET_MAIN_SOURCE_PATH \
    return QTest::qExec(&tc, argc, argv);

QTest::qExec is defined qtestcase.cpp:

int QTest::qExec(QObject *testObject, int argc, char **argv)
{
    qInit(testObject, argc, argv);
    int ret = qRun();
    qCleanup();
    return ret;
}

In qInit(), currentTestObject is set.

In qRun(), a TestMethods instance is created, and it its constructor, we find this loop:

const QMetaObject *metaObject = o->metaObject();
const int count = metaObject->methodCount();
m_methods.reserve(count);
for (int i = 0; i < count; ++i) {
     const QMetaMethod me = metaObject->method(i);
     if (isValidSlot(me))
         m_methods.push_back(me);
}

isValidSlot() is implemented thusly:

static bool isValidSlot(const QMetaMethod &sl)
{
    if (sl.access() != QMetaMethod::Private || sl.parameterCount() != 0
        || sl.returnType() != QMetaType::Void || sl.methodType() != QMetaMethod::Slot)
        return false;
    const QByteArray name = sl.name();
    return !(name.isEmpty() || name.endsWith("_data")
        || name == "initTestCase" || name == "cleanupTestCase"
        || name == "init" || name == "cleanup");
}

Finally, TestMethods::invokeMethod() is called which explicitly checks for an initTestCase first and runs it:

QTestResult::setCurrentTestFunction("initTestCase");
if (m_initTestCaseDataMethod.isValid())
    m_initTestCaseDataMethod.invoke(testObject, Qt::DirectConnection);

Likewise, it checks for cleanupTestCase at the end.

How does QTEST_MAIN set everything up automatically to run tests?

1 Answers1

Linked