How do you filter xunit tests by trait with "dotnet test"?

Question

I have a .NET Core test project that uses Xunit 2.2. Some of my tests are marked with traits.

[Fact]
[Trait("Color", "Blue")]
public void TestBlue()
{
}

What is the right command line syntax for "dotnet test" to only run tests where the trait Color == Blue ?

I'm using .NET Core CLI 1.0.0-rc4 which uses csproj, not project.json.

I'm trying to use dotnet test --filter $something, but whatever I use for $something, I see this error:

Error: [xUnit.net 00:00:00.7800155] E2ETests: Exception filtering tests: No tests matched the filter because it contains one or more properties that are not valid ($something). Specify filter expression containing valid properties (DisplayName, FullyQualifiedName) and try again.

score 77 · Accepted Answer · edited Jun 18 '20 at 06:45

77

I found the answer (only tests attributed [Trait("TraitName", "TraitValue")] are executed):

dotnet test --filter TraitName=TraitValue

Alternatively, you can filter by not having a trait value (tests attributed [Trait("TraitName", "TraitValue")] are execluded from run)

dotnet test --filter TraitName!=TraitValue

In my example above, this means I can run:

dotnet test --filter Color=Blue

More docs here: https://github.com/Microsoft/vstest-docs/blob/master/docs/filter.md

edited Jun 18 '20 at 06:45

Roi Shabtai

2,981
2
31
47

answered Feb 17 '17 at 00:00

natemcmaster

25,673
6
78
100

I still get the error about `(DisplayName, FullyQualifiedName)` any pointers on why this is broken (using cli 1.0.3) – Damian Apr 21 '17 at 00:06
Not sure, would need more details. As a workaround, you might try using `dotnet-xunit` instead. See http://xunit.github.io/docs/getting-started-dotnet-core.html – natemcmaster Apr 21 '17 at 00:08
6

The following works: `dotnet test --filter TraitName!=TraitValue` However only as long as there are some tests decorated with `[Trait("TraitName", "somevalue")]` within the test assembly... if there aren't then no tests get run :-(. Possibly a bug in dotnet test? – James Crosswell Jun 01 '17 at 14:19
For anyone running into issues using the filter to exclude tests with a particular attribute, there is currently a bug with the tooling described at https://github.com/xunit/xunit/issues/1314 – James Crosswell Jun 07 '17 at 09:11
The issue is now listed as closed (as of June 9, 2017). – Eric Pohl Aug 02 '17 at 14:35
@JamesCrosswell You are right, the only solution I have found is to include dummy Trait attribute on dummy test to test assembly without the Trait. – Karel Kral Feb 14 '19 at 12:57

score 5 · Answer 2 · answered Oct 29 '17 at 19:33

In the csproj

<PropertyGroup>
  <TargetFramework>netcoreapp2.0</TargetFramework>
</PropertyGroup>
<ItemGroup>
  <PackageReference Include="xunit" Version="2.3.0" />
  <DotNetCliToolReference Include="dotnet-xunit" Version="2.3.0" />
</ItemGroup>

command line

dotnet xunit -trait "Color=Blue"

How do you filter xunit tests by trait with "dotnet test"?

2 Answers2