Test fail when run together, passed when run individually

Question

When I implement cross browser testing with nunit using TestFixture my tests fails when run together, passed when run individually. Exception was throwed when SendKeys method was called because argument was null, but this is not the cause, because when i run this test again test will passed. Ofcourse im tried to debug this issue but i dont find solution. Simple OpenHomePage Test works fine. Here is my code:

[TestFixture(typeof(ChromeDriver))]
[TestFixture(typeof(FirefoxDriver))]
public class TestClass<TWebDriver> where TWebDriver : IWebDriver, new()
{
    [OneTimeSetUp]
    public void CreateDriver()
    {
        try
        {
            PropertiesCollection.driver = new TWebDriver();
            Console.WriteLine("Opened browser");
            PropertiesCollection.driver.Url = "http://localhost:81/";
            Console.WriteLine("Opened URL");
            PropertiesCollection.driver.Manage().Window.Maximize();
            //initialize test data from excel sheet
            ExcelLib.PopulateInCollection(@"c:\users\bolec\documents\visual studio 2015\Projects\RowingSectionTests\RowingSectionTests\TestData.xlsx");
        }
        catch (Exception msg)
        {
            Console.WriteLine(msg.ToString());
        }                                
    }

    [OneTimeTearDown]
    public void FixtureTearDown()
    {
        HomePageObjects homeObj = new HomePageObjects();
        homeObj.Logoff();
        if (PropertiesCollection.driver != null) PropertiesCollection.driver.Quit();
    }

    [TearDown]
    public void TearDown()
    {
        //Take screen on failure
        if (TestContext.CurrentContext.Result.Outcome.Status.Equals(TestStatus.Failed))
        {
            string fileName = Regex.Replace(TestContext.CurrentContext.Test.FullName + "_" + DateTime.Now.ToString(), "[^a-z0-9\\-_]+", "_", RegexOptions.IgnoreCase);
            ((ITakesScreenshot)PropertiesCollection.driver).GetScreenshot().SaveAsFile(@"c:\users\bolec\documents\visual studio 2015\Projects\RowingSectionTests\RowingSectionTests\Screenshots\" + fileName + ".png", System.Drawing.Imaging.ImageFormat.Png);
        }
    }

    //will always passed
    [Test]
    public void OpenHomePage()
    {           
        HomePageObjects homeObj = new HomePageObjects();
    }

    //login with correct credentials will login to acc
    [Test]
    public void Login()
    {            
        HomePageObjects homeObj = new HomePageObjects();
        LoginPageObjects loginObj = homeObj.ToLoginPage();
        loginObj.Login(ExcelLib.ReadData(1, "UserName"), ExcelLib.ReadData(1, "Password"));

        //checking is URL correct after loggin
        Assert.AreEqual("http://localhost:81/", PropertiesCollection.driver.Url.ToString());
        //checking is login is correct on navbar
        Assert.AreEqual(homeObj.GetUserLoginStringInButton().ToLower(), ExcelLib.ReadData(1, "UserName").ToLower());
    }

Nope. Since he is using NUnit V3 (otherwise it wouldn't compile) OneTimeSetUp and OneTimeTearDown are correct. The TestFixtureXxxxxXx attributes are deprecated in V3, but would do the same thing. — Charlie, Apr 14 '16 at 23:56
Have you enabled parallel execution? If so, the two fixtures may be interfering with one another. — Charlie, Apr 14 '16 at 23:57
Is PropertiesCollection.driver a static? If so, you have two different fixtures using the same data. Definitely won't work in parallel. If not parallel, each test needs to clean up after itself. — Charlie, Apr 14 '16 at 23:59
Yes, its static. So maybe you can steer me a little for solution? — Marcin, Apr 15 '16 at 07:39
Yes... don't use a static. :-) create a separate collection to be used in the OneTimeSetUp method of each instance of the fixture. — Charlie, Apr 16 '16 at 08:44

score 1 · Answer 1 · answered May 11 '16 at 09:17

The problem with using the static PropertiesCollection is that any changes to the static class in one test will be reflected in the other test, making the chances or creating a test dependency very high (as you have discovered).

You have two choices, firstly don't use a static instead create an instance. Alternatively, ensure that in a setup and your teardown methods you set/reset your PropertiesCollection back to it's required state.

Using the OneTimeSetUp attribute is also risky as it only runs once for all the tests in your fixture.

Test fail when run together, passed when run individually

1 Answers1