2

I want to set a continuous integration server with buildbot and gtest. I have already managed to set up the environment which leads to the following output after the unit testing step:

Running main() from gtest_main.cc
[==========] Running 7 tests from 3 test cases.
[----------] Global test environment set-up.
[----------] 4 tests from VectorTest
[ RUN      ] VectorTest.size_is_correct
[       OK ] VectorTest.size_is_correct (0 ms)
[ RUN      ] VectorTest.min_index
[       OK ] VectorTest.min_index (0 ms)
[ RUN      ] VectorTest.sort_is_correct
[       OK ] VectorTest.sort_is_correct (0 ms)
[ RUN      ] VectorTest.indices_of_smallest_are_correct
[       OK ] VectorTest.indices_of_smallest_are_correct (0 ms)
[----------] 4 tests from VectorTest (0 ms total)

[----------] 2 tests from MatrixTest
[ RUN      ] MatrixTest.NumberOfColumnsIsCorrect
[       OK ] MatrixTest.NumberOfColumnsIsCorrect (0 ms)
[ RUN      ] MatrixTest.NumberOfRowsIsCorrect
[       OK ] MatrixTest.NumberOfRowsIsCorrect (0 ms)
[----------] 2 tests from MatrixTest (0 ms total)

[----------] 1 test from SparseMatrix
[ RUN      ] SparseMatrix.IteratorIsCorrect

[       OK ] SparseMatrix.IteratorIsCorrect (0 ms)
[----------] 1 test from SparseMatrix (0 ms total)

[----------] Global test environment tear-down
[==========] 7 tests from 3 test cases ran. (2 ms total)
[  PASSED  ] 7 tests.
[100%] Built target unit

I would like buildbot to parse this output so as to check that the keyword PASSED is present in order to know if something went wrong during unit testing.

Do you know how to do that?

Aleph
  • 1,343
  • 1
  • 12
  • 27

2 Answers2

1

GoogleTest supports XML output in JUnit format using command line option --gtest_output, which most CI systems already know how to parse.

I don't know whether Buildbot supports JUnit parsing or not. If not, it is for sure easier to parse an XML structured output than the standard plain text output.

Antonio Pérez
  • 6,702
  • 4
  • 36
  • 61
  • Thank you Antonio. I will look at buildbot's documentation to see if such a parsing is possible. – Aleph Jun 08 '16 at 11:41
1

Why don't you check the exit code of the test program? It will be a success code (0) if the tests pass and a failure (usually 1) if they fail.

VladLosev
  • 7,296
  • 2
  • 35
  • 46
  • Thank you Vlad. It is not so clear for me why the exit code of the program should be a failure code when some unit test does not pass. For example, CPPUNIT does not behave like this, as far as I can remember. The program should return a failure code only when something went wrong in the program, such as an exception. But, as the program is expected to detect successes and failures in unit tests, one can't expect that it returns a failure code when it detects that a unit test failed. Actually, the program did the job well. He succeeded in detecting the test failure. – Aleph Jun 08 '16 at 11:47
  • @Aleph it is recommended to return from main() result of RUN_ALL_TESTS(): https://github.com/google/googletest/blob/master/googletest/docs/V1_5_Primer.md#invoking-the-tests So usually exit code means is all tests passed or not. But in general you are right — it's up to author of test program to decide what exit code will mean. – rutsky Jun 08 '16 at 22:23