I'm implementing a custom JUnit runner which extends BlockJUnit4ClassRunner, that can be executed using the Maven Failsafe Plugin. The runner sends the tests to a Java application server to run, then receives the test results back using Java serialization. It then manually fires off the test results to the RunNotifier given in the run() method of the class.
This is all fine, but when running using the Maven Failsafe Plugin, the plugin fails to report the correct number of tests run if there are one or more test failures; it will always report the number of tests run as equal to the number of tests failed. This happens even though the Result object sent to the RunNotifier:fireTestRunFinished() method has the correct statistics, so it doesn't look like the Failsafe plugin is directly listening to what is reported by the RunNotifier. However, if all tests pass, then the Failsafe plugin does report the correct number of tests run.
I've been looking through the Maven plugin code and it seems like they use a TestSetStats object to record the test results. However, I've been unable to understand the code further, in how the results from JUnit are fed to the TestSetStats. Can someone explain this process to me?
Furthermore, does anyone have an idea why the Maven Failsafe Plugin would not be receiving the right test results? I know the information I have given is rather general, so please feel free to ask for specifics.
