Why are my GNURadio unit tests passing when they should fail?

Question

When setting up unit testing in GNURadio (Version 3.8, running on Python 3.6.8), the assertFloatTuplesAlmostEqual method seems to give wrong results.

I'm working through the GNURadio OutOfTreeModules tutorial, focusing now on unit testing. In the unit test, I provide a vector of expected and actual results. (The code I am testing is known to be working correctly). The test uses the instruction self.assertFloatTuplesAlmostEqual(expected_result, result_data, 6) . The assert statement seems to check only the first item in the vector; if I make the expected value of any other item in the vector different than the actual, the test passes. I cannot find any documentation on the assertFloatTuplesAlmostEqual method. Here is the code for the unit test:

from gnuradio import blocks
import howto1_swig as howto1

class qa_cube_ff(gr_unittest.TestCase):

    def setUp(self):
        self.tb = gr.top_block()



    def test_001_cube_ff(self):
        print("test 001 STARTING")
        src_data = (-3, 4, -5.5 )
        print(src_data)
#        expected_result = (-27, 60, 166.375)    // these would be the correct answers
# below, only the first expected result is correct; the other 2 are intentionally wrong
        expected_result = (-27, 60, 100.375)
        src = blocks.vector_source_f(src_data)
        sqr = howto1.cube_ff()
        dst = blocks.vector_sink_f()
        self.tb.connect(src, sqr)
        self.tb.connect(sqr, dst)
        self.tb.run()
        result_data = dst.data()
        print("EXPECTED")
        print(expected_result)
        print("RESULT")
        print(result_data)
        self.assertFloatTuplesAlmostEqual(expected_result, result_data, 6)

    def tearDown(self):
        self.tb = None


if __name__ == '__main__':
    gr_unittest.run(qa_cube_ff)

Here is where I try to run the test. Note that I have put in print statements to show the expected and actual results of running the code. The assetFloatTuplesAmostEqual code tests only the first item in the vector; if the expected and actual values of the first element don't match, the test fails (as it should). Wrong values for any other element pass the test-

odroid@odroid:~/projects/grc_module/gr-howto1/build$ ctest -VV -R cube
UpdateCTestConfiguration  from :/home/odroid/projects/grc_module/gr-howto1/build/DartConfiguration.tcl
UpdateCTestConfiguration  from :/home/odroid/projects/grc_module/gr-howto1/build/DartConfiguration.tcl
Test project /home/odroid/projects/grc_module/gr-howto1/build
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 1
    Start 1: qa_cube_ff

1: Test command: /bin/sh "/home/odroid/projects/grc_module/gr-howto1/build/python/qa_cube_ff_test.sh"
1: Test timeout computed to be: 9.99988e+06
1: .
1: ----------------------------------------------------------------------
1: Ran 1 test in 0.003s
1: 
1: OK
1: test 001 STARTING
1: (-3, 4, -5.5)
1: EXPECTED
1: (-27, 60, 100.375)
1: RESULT
1: (-27.0, 64.0, -166.375)
1/1 Test #1: qa_cube_ff .......................   Passed    1.98 sec

The following tests passed:
    qa_cube_ff

100% tests passed, 0 tests failed out of 1

Total Test time (real) =   1.99 sec

Here the first element is correct (-3 cubed). The other two are wrong.

Why? A related question: Can anyone advise me where to find documentation for the gr_unittest methods??

score 0 · Answer 1 · answered Nov 03 '19 at 01:03

The assert statement seems to check only the first item in the vector; if I make the expected value of any other item in the vector different than the actual, the test passes

This sounds like a bug, consider reporting it in https://github.com/gnuradio/gnuradio/issues

I cannot find any documentation on the assertFloatTuplesAlmostEqual method. A related question: Can anyone advise me where to find documentation for the gr_unittest methods??

Sometimes the best documentation is the source code itself - https://github.com/gnuradio/gnuradio/blob/maint-3.8/gnuradio-runtime/python/gnuradio/gr_unittest.py#L104-L123

def assertFloatTuplesAlmostEqual(self, a, b, places=7, msg=None):
    """
    Fail if the two real-valued tuples are not approximately equal.
    Approximate equality is determined by specifying the number of decimal
    places.
    """
    self.assertEqual(len(a), len(b))
    return all((
        self.assertAlmostEqual(x, y, places, msg)
        for (x, y) in zip(a, b)
    ))


def assertFloatTuplesAlmostEqual2(self, a, b,
                                  abs_eps=1e-12, rel_eps=1e-6, msg=None):
    self.assertEqual(len(a), len(b))
    return all((
        self.assertComplexAlmostEqual2(x, y, abs_eps, rel_eps, msg)
        for (x, y) in zip(a, b)
    ))

A link to your bug report https://github.com/gnuradio/gnuradio/issues/2883 — Vasil Velichkov, Nov 04 '19 at 23:55

Why are my GNURadio unit tests passing when they should fail?

1 Answers1