0

In response of another SO questions, I came across a problem when running async Task with Xunit and visual studio 2015 ctp6.

here is the code:

    using System;
    using System.Threading.Tasks;
    using Microsoft.AspNet.TestHost;
    using Microsoft.Framework.DependencyInjection;
    using Xunit;
    using Microsoft.AspNet.Builder;
    using System.Net.Http;

    namespace Multi.Web.Api
    {
        public class TestServerHelper : IDisposable
        {
            public TestServerHelper()
            {
                ClientProvider = new TestClientProvider();

                ApiServer = TestServer.Create((app) =>
                {
                    app.UseServices(services =>
                    {
                        services.AddTransient<IClientProvider>(s => ClientProvider);
                    });
                    app.UseMulti();
                });
            }
            public TestClientProvider ClientProvider { get; private set; }

            public TestServer ApiServer { get; private set; }

            public void Dispose()
            {
                ApiServer.Dispose();
                ClientProvider.Dispose();
            }
        }

        public class MultiMiddlewareTest : IClassFixture<TestServerHelper>
        {

            TestServerHelper _testServerHelper;

            public MultiMiddlewareTest(TestServerHelper testServerHelper)
            {
                _testServerHelper = testServerHelper;

            }

            [Fact]
            public async Task ShouldReturnToday()
            {
                using (HttpClient client = _testServerHelper.ApiServer.CreateClient())
                {
                    var response = await client.GetAsync("http://localhost/today");

                    String content = await response.Content.ReadAsStringAsync();
                    Assert.Equal(content, "2015-04-15 count is 1");
                }
            }

            [Fact]
            public async Task ShouldReturnYesterday()
            {
                using (HttpClient client = _testServerHelper.ApiServer.CreateClient())
                {
                    var response = await client.GetAsync("http://localhost/yesterday");

                    String content = await response.Content.ReadAsStringAsync();
                    Assert.Equal(content, "2015-04-14 count is 1");
                }
            }
        }
    }

in visual studio TestExplorer, when running the test one by one (right click and select debug selected test) it's ok, but when running all, none of the passes and I have the following error

Message : Response status code does not indicate success : 404 (Not Fount)

all the code is available on the other so question, in that question, I answered on how to use multiple instance of TestServer to mock external Api. And I think it has to do with some Synchronization context.

I think I wrote my Helper not in a good way because I see it disposes objects before the call is actually done (sometimes not...). does someone had the same issue and had a solution on this ?

UPDATE : link to full code on github

Community
  • 1
  • 1
Cedric Dumont
  • 1,009
  • 17
  • 38
  • Are u using xunit.runner.kre or xunit.runnner.aspnet. There's an issue in the kre runner: https://github.com/xunit/aspnet.xunit/issues/20. But there's also an issue using aspnet runner: VS 2015 CT6 doesn't discover tests – agua from mars Apr 15 '15 at 19:08
  • i am using xunit.runnner.aspnet. and for the Test that are not discoverred, If you name your file the same name as your class it's ok. so in file MyTest.cs, the classname must be MyTest. if you have a different name I saw that VS TestExplorer is looping infinitely and try to discover but discovers nothing. – Cedric Dumont Apr 16 '15 at 05:12

1 Answers1

4

Xunit is running your tests in parallel by default when you run all tests; I'm guessing this is probably causing your test servers to collide and have side effects. (I'm not sure what all packages you're using in your project.json, so I could be mistaken.) Take a look at Running tests in parallel in the Xunit documentation and find the solution right for you.

Options:

  1. Use the CollectionAttribute such as [Collection("Localhost http server")].
  2. Specify -parallel none as the option on the command line.
  3. Change the default behavior using an assembly attribute such as [assembly: CollectionBehavior(DisableTestParallelization = true)]

UPDATE 2:

Update 1 was a red herring, so I removed it entirely. Removing await next(context); from the FakeExternalApi middleware seems to have removed the intermittent issues. A comment from @Tratcher indicates that "calling Next ... is discouraged," but I'm not sure if that's specifically related to OwinMiddleware or if it is good general advice, but it seems to apply here.

Community
  • 1
  • 1
Matt DeKrey
  • 11,582
  • 5
  • 54
  • 69
  • In fact I see that if the test method are in the same class they are not run in parallel. Nevertheless, I tried to do it with these comments, but it didn't work.I'll wait untill some xunit libraries are no more in beta. – Cedric Dumont Apr 16 '15 at 06:18
  • 1
    @CedricDumont, I've updated my answer given your full source code. Thanks for providing it! – Matt DeKrey Apr 17 '15 at 13:21
  • I commited the changes on the repo, but it did'nt solve the issue. note : I am using visual studion ctp6. perhaps it comes from there. – Cedric Dumont Apr 17 '15 at 13:42
  • Oh, wow, it's intermittent; if I run without debugging, it usually goes fine. Looking more! – Matt DeKrey Apr 17 '15 at 14:08
  • Hopefully I've finally nailed down your issue, @CedricDumont. It's been an interesting one to debug; I hadn't made any middleware myself yet. – Matt DeKrey Apr 17 '15 at 14:43
  • yes , this solves the problem. but wonder why.... I will also wait the final release of asp.net 5 to test it again – Cedric Dumont Apr 20 '15 at 07:37