0

I have recently started a new project and decided to use Specflow 2.1 It ships with NUnit3.

Currently the project is setup and the test runner in Visual studio is executing correctly however when I try and run the same tests from the Nunit 3 console runner I get an error.

The command I use is nunit3-console.exe --labels=All --framework=net-4.5 Blah.Testing.Specflow.dll

I added two native Nunit tests with the same assembly and those are executing fine, its the one specflow test that is failing

NUnit Console Runner 3.2.1
Copyright (C) 2016 Charlie Poole

Runtime Environment
   OS Version: Microsoft Windows NT 10.0.10586.0
  CLR Version: 4.0.30319.42000

Test Files
    Blah.Testing.Specflow.dll

=> Blah.Testing.Specflow.MyTest.TestOne
=> Blah.Testing.Specflow.MyTest.TestTwo
=> Blah.Testing.Specflow.MyTestFeature.AddTwoNumbers

Errors and Failures

1) TearDown Error : Blah.Testing.Specflow.MyTestFeature
System.ArgumentNullException : Value cannot be null.
Parameter name: instance
TearDown : System.NullReferenceException : Object reference not set to an instance of an object.
   at BoDi.ObjectContainer.RegisterInstanceAs(Object instance, Type interfaceType, String name, Boolean dispose)
   at TechTalk.SpecRun.SpecFlowPlugin.Runtime.SpecRunTestRunnerManager.CreateTestRunnerInstance() in c:\TeamCity\BuildAgent\work\245a3e4d646c0875\SpecFlowPlugins\TechTalk.SpecRun.SpecFlowPlugin.2-0-0\Runtime\SpecRunTestRunnerManager.cs:line 33
   at TechTalk.SpecFlow.TestRunnerManager.CreateTestRunner(Int32 threadId)
   at TechTalk.SpecFlow.TestRunnerManager.GetTestRunner(Int32 threadId)
   at TechTalk.SpecFlow.TestRunnerManager.GetTestRunner(Assembly testAssembly, Nullable`1 managedThreadId)
   at Blah.Testing.Specflow.MyTestFeature.FeatureSetup() in D:\Projects\Blah\src\Testing\Blah.Testing.Specflow\MyTest.feature.cs:line 0
--TearDown
   at Blah.Testing.Specflow.MyTestFeature.FeatureTearDown() in D:\Projects\Blah\src\Testing\Blah.Testing.Specflow\MyTest.feature.cs:line 0

2) Error : Blah.Testing.Specflow.MyTestFeature.AddTwoNumbers
OneTimeSetUp: System.ArgumentNullException : Value cannot be null.
Parameter name: instance

Run Settings
    RuntimeFramework: net-4.5
    WorkDirectory: D:\Projects\Blah\src\Testing\Blah.Testing.Specflow\bin\Dev
    ImageRuntimeVersion: 4.0.30319
    ImageTargetFrameworkName: .NETFramework,Version=v4.5.1
    ImageRequiresX86: False
    ImageRequiresDefaultAppDomainAssemblyResolver: False
    NumberOfTestWorkers: 8

Test Run Summary
  Overall result: Failed
  Test Count: 3, Passed: 2, Failed: 1, Inconclusive: 0, Skipped: 0
    Failed Tests - Failures: 0, Errors: 1, Invalid: 0
  Start time: 2016-05-16 00:52:52Z
    End time: 2016-05-16 00:52:53Z
    Duration: 0.380 seconds

Results (nunit3) saved as TestResult.xml

I have tried reflecting the TechTalk.Specflow Assembly to go look at the CreateTestRunnerInstance method however didnt find anything useful to guide me to a solution.

The generated step file code is

// ------------------------------------------------------------------------------
//  <auto-generated>
//      This code was generated by SpecFlow (http://www.specflow.org/).
//      SpecFlow Version:2.1.0.0
//      SpecFlow Generator Version:2.0.0.0
// 
//      Changes to this file may cause incorrect behavior and will be lost if
//      the code is regenerated.
//  </auto-generated>
// ------------------------------------------------------------------------------
#region Designer generated code
#pragma warning disable
namespace Blah.Testing.Specflow
{
    using TechTalk.SpecFlow;


    [System.CodeDom.Compiler.GeneratedCodeAttribute("TechTalk.SpecFlow", "2.1.0.0")]
    [System.Runtime.CompilerServices.CompilerGeneratedAttribute()]
    [NUnit.Framework.TestFixtureAttribute()]
    [NUnit.Framework.DescriptionAttribute("MyTest")]
    public partial class MyTestFeature
    {

        private TechTalk.SpecFlow.ITestRunner testRunner;

#line 1 "MyTest.feature"
#line hidden

        [NUnit.Framework.TestFixtureSetUpAttribute()]
        public virtual void FeatureSetup()
        {
            testRunner = TechTalk.SpecFlow.TestRunnerManager.GetTestRunner();
            TechTalk.SpecFlow.FeatureInfo featureInfo = new TechTalk.SpecFlow.FeatureInfo(new System.Globalization.CultureInfo("en-US"), "MyTest", "\tIn order to avoid silly mistakes\r\n\tAs a math idiot\r\n\tI want to be told the sum o" +
                    "f two numbers", ProgrammingLanguage.CSharp, ((string[])(null)));
            testRunner.OnFeatureStart(featureInfo);
        }

        [NUnit.Framework.TestFixtureTearDownAttribute()]
        public virtual void FeatureTearDown()
        {
            testRunner.OnFeatureEnd();
            testRunner = null;
        }

        [NUnit.Framework.SetUpAttribute()]
        public virtual void TestInitialize()
        {
        }

        [NUnit.Framework.TearDownAttribute()]
        public virtual void ScenarioTearDown()
        {
            testRunner.OnScenarioEnd();
        }

        public virtual void ScenarioSetup(TechTalk.SpecFlow.ScenarioInfo scenarioInfo)
        {
            testRunner.OnScenarioStart(scenarioInfo);
        }

        public virtual void ScenarioCleanup()
        {
            testRunner.CollectScenarioErrors();
        }

        [NUnit.Framework.TestAttribute()]
        [NUnit.Framework.DescriptionAttribute("Add two numbers")]
        [NUnit.Framework.CategoryAttribute("mytag")]
        public virtual void AddTwoNumbers()
        {
            TechTalk.SpecFlow.ScenarioInfo scenarioInfo = new TechTalk.SpecFlow.ScenarioInfo("Add two numbers", new string[] {
                        "mytag"});
#line 7
this.ScenarioSetup(scenarioInfo);
#line 8
 testRunner.Given("I am on the homepage", ((string)(null)), ((TechTalk.SpecFlow.Table)(null)), "Given ");
#line 9
testRunner.Then("I set the Implicit Wait to \"5000\"ms", ((string)(null)), ((TechTalk.SpecFlow.Table)(null)), "Then ");
#line 10
testRunner.Then("I take a screenshot called \"\"", ((string)(null)), ((TechTalk.SpecFlow.Table)(null)), "Then ");
#line hidden
            this.ScenarioCleanup();
        }
    }
}
#pragma warning restore
#endregion

Any help will be greatly appreciated

Ettienne
  • 365
  • 3
  • 9
  • can you provide the code from your steps files please? and also the step which is attributed with `[OneTimeSetUp]` if you have one – Sam Holder May 16 '16 at 07:33
  • I have added the generated file. Doesnt appear to be generating a OneTimeSetup Attrib on any of the generated code. Its worth noting I am using the nunit test provider – Ettienne May 17 '16 at 05:15

1 Answers1

0

It looks like, you have the SpecFlow+Runner (SpecRun) plugin also configured.

at TechTalk.SpecRun.SpecFlowPlugin.Runtime.SpecRunTestRunnerManager.CreateTestRunnerInstance() in c:\TeamCity\BuildAgent\work\245a3e4d646c0875\SpecFlowPlugins\TechTalk.SpecRun.SpecFlowPlugin.2-0-0\Runtime\SpecRunTestRunnerManager.cs:line 33

please check your plugins and unittestprovider setting in the app.config. After that, regenerate all feature- code- behind files (simple save the feature- files).

Andreas Willich
  • 5,665
  • 3
  • 15
  • 22
  • Hi @Andreas thank you so much for point this out it worked! I did add the NUnit test provider but left the SpecRun plugin in place (I was under the impression this was only for VS running and would not affect the generated assembly) When I removed the Specrun plugin section and regenerated the backing code it all started working :) – Ettienne May 31 '16 at 23:18
  • How did you go about the problem? I have very similar issue. Whenever I run the tests it always drops the exception in SpecFlowNUnitExtension.cs file and this drops me mad. The following is set in my config file: with SpecRun being set as Test Runner Tool in settings. – Jakubee Jun 01 '16 at 10:15
  • @Jakubee Would you please open an issue about it here: https://github.com/techtalk/SpecFlow To solve your problem I need more information about your config and this will be to much for the comment fields here. Thanks! – Andreas Willich Jun 01 '16 at 10:25