21

I have integration tests set up with xUnit.net.

Is there a way to configure how long an integration test should last maximum? I mean a threshold.

Monika
  • 2,172
  • 15
  • 24
codefruit
  • 388
  • 1
  • 2
  • 7
  • 1
    Probably not available at the time, but `dotnet test --blame-hang-timeout 30s` would be another way to disallow any single test to run longer than 30 seconds. It is not controlled by xUnit, but by the test runner, and can be used with any framework and doesn't suffer the issues with `Fact(Timeout=30000ms)`, which xUnit most of the time just ignores... – Abel Oct 16 '22 at 16:59

4 Answers4

16

For the newer xunit versions, I am using this method, which seems to work well:

public static class AssertAsync
{
    public static void CompletesIn(int timeout, Action action)
    {
        var task = Task.Run(action);
        var completedInTime = Task.WaitAll(new[] { task }, TimeSpan.FromSeconds(timeout));

        if (task.Exception != null)
        {
            if (task.Exception.InnerExceptions.Count == 1)
            {
                throw task.Exception.InnerExceptions[0];
            }

            throw task.Exception;
        }

        if (!completedInTime)
        {
            throw new TimeoutException($"Task did not complete in {timeout} seconds.");
        }
    }
}

You can use it like so:

[Fact]
public void TestMethod()
{
    AssertAsync.CompletesIn(2, () =>
    {
        RunTaskThatMightNotComplete();
    });
}

However, be sure to read the reason about why this was removed in the first place as one of the maintainers of the project does make a pretty good point about potentially deadlocking your test run. I run these tests in single-threaded mode and it doesn't appear to cause a problem.

Steve Rukuts
  • 9,167
  • 3
  • 50
  • 72
  • 5
    Looks like xUnit again supports [`[Fact(Timeout=milliseconds)]`](https://github.com/xunit/xunit/blob/42031495bc52d10d52733e4af9ea33a8c1257a05/src/xunit.v3.core/FactAttribute.cs#L32), but only for non-parallel runs. – Jarrod Dixon Apr 04 '20 at 16:18
13

It seems, what you are looking for is the Timeout parameter of the Fact attribute.

For further information see the XUnit Docs under Ensuring a Test Does Not Run Too Long.

Chris McKee
  • 4,298
  • 10
  • 48
  • 83
bbohac
  • 333
  • 2
  • 12
  • 5
    The answer is not helpful at all. The link leads to a documentation page, but not to a specifc article. While, the specific article on the `Fact` parameters seems to be absent in the documentation whatsoever. – hellouworld Jan 20 '20 at 10:12
  • 1
    I believe that it is necessary to add a link to a specific article from the documentation on the `Fact` and add an excerpt from the article here. – hellouworld Jan 20 '20 at 10:13
  • 1
    WARNING: Using this with parallelization turned on will result in undefined behavior. Timeout is only supported when parallelization is disabled, either globally or with a parallelization-disabled test collection. – Shoter Jun 13 '21 at 16:26
11

Unfortunately, in newer versions it looks like the Fact attribute no longer has a Timeout parameter

Tobias J
  • 19,813
  • 8
  • 81
  • 66
Tim
  • 412
  • 7
  • 18
  • Do you know if there are any extensions that add this feature back in? – LWood Jul 29 '16 at 14:24
  • Not that I know of. I am not very familiar with it but reading a discussion thread it was removed for purity. The argument being that with thread scheduling and such it was impossible to guarantee it was accurate. – Tim Aug 16 '16 at 15:27
  • 3
    I think they brought it back in 2.4 but for parallel execution, it's behavior is "undefined" – Ronnie Overby Aug 30 '18 at 14:52
7

To get a concrete example:

You can just use

[Fact(Timeout = 2000)]

for example.

Hint: Timeout is specified in ms.

Brandon Minnick
  • 13,342
  • 15
  • 65
  • 123
holbizmetrics
  • 81
  • 1
  • 2