1

I'm going through an exercise to build some integration tests using specflow, and view some test results after the execution of these tests.

I'm seeing some differences in the quality of the report generated by the specflow executable between test results generated from nunit and mstest.

The mstest report seems to be more in keeping with what I want to see, however we as a company use nunit, and I would prefer to use nunit because of this, while generating the same report as we see from mstest.

The lacking report that we see from nunit looks like nunit report

The more desired report that we see from mstest looks like mstest report

The configuration I'm using for the specflow project is

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <configSections>
    <section name="specFlow" type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow" />
  </configSections>
  <specFlow>

    <trace 
        traceSuccessfulSteps="true" 
        traceTimings="true" 
        minTracedDuration="0:0:0.1" 
        stepDefinitionSkeletonStyle="RegexAttribute" />
    <unitTestProvider name="nunit" /> <!-- delete as appropriate -->
    <!-- unitTestProvider name="mstest" / -->
  </specFlow>
</configuration>

The command line arguments I'm running are:
mstest:

MSTest.exe /testcontainer:%testAssembly% /resultsfile:%resultsTrx% /detail:testname
specflow.exe mstestexecutionreport %testProject% /testresult:%resultsTrx% /out:%resultsOut%

nunit:

nunit-console.exe /labels /out=%resultsTxt% /xml=%resultsXml% %testAssembly%
specflow.exe  nunitexecutionreport %testProject% /xmlTestResult:%resultsXml% /testOutput:%resultsTxt% /out:%resultsOut%

What can I do to align these reports?

J Robson
  • 149
  • 2
  • 12
  • If you look into [the sources of the specflow binary](https://github.com/techtalk/SpecFlow) you'll see that the report is formatted by a xslt template. You can tweak the template to get your custom formatting. Make sure all data is present in the resultsXml. The newer VS testrunner, VSTest.Console.exe lack certain information in the .trx export for example. Maybe you are experiencing the same issues. – AutomatedChaos Jul 27 '15 at 19:28
  • 1
    Why do you care what framework Specflow uses under the hood? If you prefer the MSTest report, why not just configure spwecflow to use MSTest. If this is some extra effort on your build servers, weigh this work up against the work customising the report in nunit... – Sam Holder Jul 27 '15 at 22:18

0 Answers0