How to select a tests form a test suite

Question

I have a stage with the follow tasks:

There are 2 test suites with 8 test cases. I want to execute only test cases with tag "download" in feature file

When I locally execute command

dotnet test <name>.dll --filter TestCategory=download

everything is fine, but when I try to add command-line options in "Other console options" in task test cases are not filtered and all are executed.

What am I doing wrong and what can help me to filter tests?

Hi @Artsiom. Is there any update about this ticket? Feel free to let me know if the answers could meet your requirement. You may consider [accepting the useful one as answer](https://meta.stackexchange.com/questions/5234/how-does-accepting-an-answer-work). — Kevin Lu-MSFT, Aug 26 '20 at 08:48

score 1 · Answer 1 · answered Aug 24 '20 at 18:55

1

The other console options are not honored when running from a test plan. You either need to change it to select tests using an assembly (which matches your local copy) or you need to filter the tests in the suite using a filter criteria not tied in source code.

If you expand the info of that option:

These options are not supported and will be ignored when running tests using the ‘Multi agent’ parallel setting of an agent job or when running tests using ‘Test plan’ option. The options can be specified using a settings file instead.

answered Aug 24 '20 at 18:55

Matt

3,658
3
14
27

Thanks for the answer, Matt! Can you tell me how not to use a settings file in this case? (the project created with .NET Core 3.1) – Artsiom Aug 24 '20 at 19:40
I don't think TestCaseFilter is one of the properties allowed in the runsettings file. You would probably need to either filter your suite or run against the specific assembly instead. https://learn.microsoft.com/en-us/visualstudio/test/configure-unit-tests-by-using-a-dot-runsettings-file?view=vs-2019&redirectedfrom=MSDN – Matt Aug 24 '20 at 20:04

score 1 · Answer 2 · answered Aug 25 '20 at 08:15

Matt is correct. The other console options in Visual Studio test task doesn't support running from a test plan.

To solve this issue, you could use .runsettings file.

Now, versions after VS 16.6 preview 3 can support adding testcasefilter to .runsettings directly.

Here is an example:

.runsettings file

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
  <!-- Configurations that affect the Test Framework -->
  <RunConfiguration>
    <MaxCpuCount>1</MaxCpuCount>
    <!-- Path relative to directory that contains .runsettings file-->
    <ResultsDirectory>.\TestResults</ResultsDirectory>
    <TestCaseFilter>TestCategory=xxx</TestCaseFilter>
  </RunConfiguration>
....

Pipeline Settings:

You could install the VS 16.7.1 in Visual Studio Test Platform Installer task.

Then you could set the Test Platform version and runsettings file in the visual studio test task.

Here is a ticket about testcasefilter in runsettings file.

How to select a tests form a test suite

2 Answers2