2

Using nunit 2.6.4 and AutoMoqData the Resharper runner appears to be evaluating all of the parameters to be passed into all tests prior to executing a single test, even if all I want to do is run a single test/small suite of tests. Right now (we have 1000's of tests) it's taking 2-3 minutes to run a single test, which doesn't work for TDD.

I tried switching to Xunit to see if nunit was the issue and there was still a big delay before running the first test.

Is this behaviour to be expected? Or are we doing something wrong?

Carl
  • 1,782
  • 17
  • 24
  • 5
    Are you able to share a solution that demonstrates the problem? One we could download locally and profile? Does it happen with the console runners? – Adam C May 25 '15 at 05:08
  • 1
    AKA [SSCCE](http://sscce.org). – Mark Seemann May 25 '15 at 09:02
  • 1
    This isn't expected. I have various test suites using AutoFixture and AutoMoqData running about a thousand tests in less than ten seconds on an ultrabook. – Mark Seemann May 25 '15 at 09:04
  • The domain is a fairly complicated EF graph, and my primitive profiling is pointing to AutoMoqData creating lots and lots of objects as the bottleneck here. I added very simple registrations for each object (about 100 of them) which didn't make any difference, but the primitive profiling showed that the majority of the time was spent creating the IFixtures. Now I'm using a singleton fixture (with the registrations) and that has brought the time to test down to a more reasonable 8s. For ref, using a singleton fixture without the registrations didn't make much difference to the original timings. – Carl May 25 '15 at 13:59
  • An issue with using a single fixture is that any frozen services are forever frozen. Back to the drawing board... – Carl May 25 '15 at 21:03
  • "a fairly complicated EF graph" ...One thing I've noticed about AutoFixture is that if you use it for big object graphs, it tends to slow down. If your graphs are big enough, that could be the explanation. It's not particularly *expected* behaviour, but AutoFixture (being a TDD tool) was never designed to build *large* graphs. How big are your graphs? – Mark Seemann Jun 02 '15 at 20:06
  • Too big :) I've worked it down and I'm only creating the object and one level down, with empty collections and it's made a huge difference, back to 8s. – Carl Jun 03 '15 at 08:16
  • The root of my problems is the design of the app and the API. I tried to ignore your warnings @Mark, but in the end it just means more work to refactor/redesign the app. A good learning experience though :) – Carl Jun 03 '15 at 08:28
  • In looking at a remarkably similar problem to @Carl 's, AutoFixture.Kernel.RecursionGuard is 8.46% time of the total test. – John Zabroski Apr 29 '21 at 20:34

1 Answers1

1

So the results of my investigation are that when nunit discovers the tests it runs through the attributes and creates the objects, and nunit (2) discovers all the tests, even if you are only interested in running 1. Apparently this will change at some point for nunit 3.

The complicated and large object graph was the reason that the tests were slowing down, and by customising Autofixture to brutally prune this graph the tests are now much much (260s - 8s) faster.

I tried using Autofixture.AutoEntityFramework, but although it was doing what I wanted it to do, the speed gains were not enough to effectively TDD (260s - about 100s).

Carl
  • 1,782
  • 17
  • 24