1

Why the following xUnit test doesn't get passed. ProgramFixture constructor is running 3 times, but I just want to keep the same instance for all tests, so the Count property should be: 1, 2, 3 in a sequence. Why it's giving me: 1, 1, 1 as it's instantiating a new ProgramFixture for every InlineData test.

Program:

namespace UnitTest
{
    public sealed class Program
    {
        public int Count { get; set; }

        public void IncrementCount()
        {
            ++this.Count;
        }

        // Mandatory Main method for the entry point.
        public static void Main() { }
    }
}

xUnit:

using System;
using System.Diagnostics;
using Xunit;

namespace UnitTestTests
{
    public sealed class ProgramFixture : IDisposable
    {
        private bool disposed = false;

        public UnitTest.Program Program { get; }

        public ProgramFixture()
        {
            this.Program = new();
            Debug.WriteLine("################## ProgramFixture constructor runned.");
        }

        public void Dispose()
        {
            Dispose(true);
            GC.SuppressFinalize(this);
        }

        private void Dispose(bool disposing)
        {
            if (this.disposed)
                return;

            if (disposing)
            {
                Debug.WriteLine("################## ProgramFixture disposed.");
            }

            this.disposed = true;
        }
    }

    public sealed class UnitTest1 : IClassFixture<ProgramFixture>
    {
        private readonly ProgramFixture programFixture;

        public UnitTest1(ProgramFixture programFixture)
        {
            this.programFixture = programFixture;
        }

        [Theory]
        [InlineData(1)]
        [InlineData(2)]
        [InlineData(3)]
        public void Test1(int expectedCount)
        {
            this.programFixture.Program.IncrementCount();

            Assert.Equal(expectedCount, this.programFixture.Program.Count);
        }
    }
}

Error: enter image description here

When debugging it: enter image description here

I'm using Visual Studio Community 2019 16.10.3

Is that a bug on Visual Studio or some machine specific bug?

EDIT

My Project files:

Program:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net5.0</TargetFramework>
  </PropertyGroup>

</Project>

xUnit:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net5.0</TargetFramework>

    <IsPackable>false</IsPackable>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.9.4" />
    <PackageReference Include="xunit" Version="2.4.1" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.4.3">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
    <PackageReference Include="coverlet.collector" Version="3.0.2">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
  </ItemGroup>

  <ItemGroup>
    <ProjectReference Include="..\UnitTest\UnitTest.csproj" />
  </ItemGroup>

</Project>

EDIT 2

I noticed I have inverted the Assert.Equal method signature order for its parameters, so I edited Assert.Equal(this.programFixture.Program.Count, expectedCount) to Assert.Equal(expectedCount, this.programFixture.Program.Count) so in the first Error screenshot the expected and actual values shown at the right panel are different from this one, however the problem still the same.

enter image description here

Simple
  • 827
  • 1
  • 9
  • 21
  • I've just tried this, and I don't see the behavior you're describing - I *do* see counts of 1, 2 and 3, but with the assertions not necessarily in the order that the inline test data is specified. – Jon Skeet Jul 10 '21 at 06:15
  • @JonSkeet I did update the question with more info, it's weird because in my machine they don't get passed. If you reproduced the same way so it makes me think some kind of bug however my `Visual Studio Community 2019` is up to date. – Simple Jul 10 '21 at 06:35
  • Note that you're showing a success for your "3" test - which suggests in that run it's just the first two that are the wrong way round. (It's also slightly confusing because you've got the expected and actual arguments the wrong way round in your test.) But when I debug through the code, it only shows the constructor running once. Perhaps include your project file so we can see your dependency versions? – Jon Skeet Jul 10 '21 at 06:40
  • @JonSkeet As you requested question has been updated in the EDIT section. Also noticed that sometimes the constructor runs 1 time as you described, but if you run the test more times it will eventually run 2 or even 3 as in the screenshot. – Simple Jul 10 '21 at 06:56
  • 1
    I've just run it several times, and not seen it happen once. I don't know what's going on on your machine, I'm afraid :( – Jon Skeet Jul 10 '21 at 07:07
  • @JonSkeet Looks like I'm not the only one having the same issue: github.com/xunit/xunit/issues/120 – Simple Jul 10 '21 at 17:50
  • That was 7 years ago, and fixed in VS 2013... I think it unlikely to actually be the same cause. – Jon Skeet Jul 10 '21 at 18:05
  • @JonSkeet Bug confirmed: https://github.com/xunit/xunit/issues/2347 – Simple Feb 13 '23 at 18:05

1 Answers1

0

Bug confirmed and bug report has been filled on https://github.com/xunit/xunit/issues/2347. Unfortunately I'm forced to use a less idiomatic unit test approach but at least it's robust, reliable and works without none of the issues mentioned, and fits for the purpose of another object attributes other than Count to preserve its same state during this single test context:

Program:

using System;
using System.Diagnostics;

namespace UnitTest
{
    public sealed class Program : IDisposable
    {
        public int Count { get; set; }

        private bool disposed = false;

        public Program()
        {
            Debug.WriteLine("################## Program constructor runned.");
        }

        public void IncrementCount()
        {
            ++this.Count;
        }

        public void Dispose()
        {
            Dispose(true);
            GC.SuppressFinalize(this);
        }

        private void Dispose(bool disposing)
        {
            if (this.disposed)
                return;

            if (disposing)
            {
                Debug.WriteLine("################## Program disposed.");
            }

            this.disposed = true;
        }

        // Mandatory Main method for the entry point.
        public static void Main() { }
    }
}

xUnit:

using Xunit;

namespace UnitTestTests
{
    public sealed class UnitTest1
    {
        [Fact]
        public void Test1()
        {
            using (var program = new UnitTest.Program())
            {
                int[] expectedCounts = { 1, 2, 3 };

                foreach (var expectedCount in expectedCounts)
                {
                    program.IncrementCount();
                    Assert.Equal(expectedCount, program.Count);
                }
            }
        }
    }
}
Simple
  • 827
  • 1
  • 9
  • 21