How can I disable DUnit tests by name programmatically?

Question

For integration tests, I created a DUnit test suite which runs once for every version of a third party component (a message broker). Unfortunately, some tests always fail because of known bugs in some versions of the tested component.

This means the test suites will never complete with 100%. For automated tests however, a 100% success score is required. DUnit does not offer a ready-made method to disable tests in a test suite by name.

score 7 · Accepted Answer · edited Nov 27 '10 at 00:15

7

I wrote a procedure which takes a test suite and a list of test names, disables all tests with a matching name, and also performs a recursion into nested test suites.

procedure DisableTests(const ATest: ITest; const AExclude: TStrings);
var
  I: Integer;
begin
  if AExclude.IndexOf(ATest.Name) <> -1  then
  begin
    ATest.Enabled := False;
  end;
  for I := 0 to ATest.Tests.Count - 1 do
  begin
    DisableTests(ATest.Tests[I] as ITest, AExclude);
  end
end;

Example usage (the TStringlist ‘Excludes’ is created in the Setup method):

procedure TSuiteVersion1beta2.SetUp;
begin
  // fill test suite
  inherited;

  // exclude some tests because they will fail anyway
  Excludes.Add('TestA');
  Excludes.Add('TestB');

  DisableTests(Self, Excludes);
end;

edited Nov 27 '10 at 00:15

Warren P

65,725
40
181
316

answered Nov 25 '10 at 16:32

mjn

36,362
28
176
378

btw, I would love to see how you register your tests and how you get a reference to the component for each specific testsuite in your testcase. – Lieven Keersmaekers Nov 25 '10 at 18:13
maybe OpenCTF (open component test framework for Delphi) gives some ideas, it generates suites of unit tests for every single component and validates their properties - http://sourceforge.net/projects/openctf/ – mjn Nov 25 '10 at 20:24
Very nice! I actually needed this too. – Warren P Nov 27 '10 at 00:15

yonojoy · Answer 2 · 2023-05-24T08:29:07.687

For tests, that are known to fail, you can implement ITestListener interface and override ShouldRunTest.

The advantage of this solution (compared to DisableTests by @WarrenP) is, that it does not affect the check boxes, that determine which tests to run.

I use a common base class TMvTestSuite for my test suites, that adds a function AddKnownInvalidTest that can be called in the constructor (e.g. after AddTest):

type

    /// Listener that only implements ShouldRunTest
    TIShouldRunTestListener = class(TInterfacedObject, ITestListener)
    private
        FInvalidTests: TStrings;
        function IsInvalidTest(ATest: ITest): Boolean;
    protected
        function  ShouldRunTest(ATest: ITest):Boolean;
        procedure Status(test :ITest; const Msg :string);
        procedure TestingStarts;
        procedure StartTest(test: ITest);
        procedure AddSuccess(test: ITest);
        procedure AddError(error: TTestFailure);
        procedure AddFailure(Failure: TTestFailure);
        procedure EndTest(test: ITest);
        procedure TestingEnds(testResult :TTestResult);
    public
        constructor Create(AInvalidTests: TStrings);
    end;

    TMvTestSuite = class abstract(TTestSuite, ITestSuite, ITest)
    private
        FInvalidTests: TStrings;
    protected
        procedure RunTest(TestResult: TTestResult); override;
    protected
        procedure AddKnownInvalidTest(ATestName: string);
    public
        destructor Destroy; override;
    end;

implementation

constructor TIShouldRunTestListener.Create(AInvalidTests: TStrings);
begin
    FInvalidTests := AInvalidTests;
end;

function TIShouldRunTestListener.IsInvalidTest(ATest: ITest): Boolean;
begin
    Result := Assigned(FInvalidTests)
      and (FInvalidTests.IndexOf(ATest.Name) <> -1);
end;

function TIShouldRunTestListener.ShouldRunTest(ATest: ITest): Boolean;
begin
    Result := not IsInvalidTest(ATest);
end;

procedure TIShouldRunTestListener.AddError(error: TTestFailure);
begin
end;

procedure TIShouldRunTestListener.AddFailure(Failure: TTestFailure);
begin
end;

procedure TIShouldRunTestListener.AddSuccess(test: ITest);
begin
end;

procedure TIShouldRunTestListener.EndTest(test: ITest);
begin
end;

procedure TIShouldRunTestListener.StartTest(test: ITest);
begin
end;

procedure TIShouldRunTestListener.Status(test: ITest; const Msg: string);
begin
end;

procedure TIShouldRunTestListener.TestingEnds(testResult: TTestResult);
begin
end;

procedure TIShouldRunTestListener.TestingStarts;
begin
end;

{ TMvTestSuite }

destructor TMvTestSuite.Destroy;
begin
    FreeAndNil(FInvalidTests);
    inherited;
end;

procedure TMvTestSuite.AddKnownInvalidTest(ATestName: string);
begin
    if not Assigned(FInvalidTests) then
      FInvalidTests := TStringList.Create;

    FInvalidTests.Add(ATestName);
end;


procedure TMvTestSuite.RunTest(TestResult: TTestResult);
begin
    TestResult.AddListener(TIShouldRunTestListener.Create(FInvalidTests));
    inherited;
end;

How can I disable DUnit tests by name programmatically?

2 Answers2

Linked