22

I'd like to reduce the time which our build (using ant) takes for running the tests. Currently I am using the default forkMode, which forks a new vm on each test class (perTest).

I am thinking about to switch to forkMode="once" but I am unsure if this will couple the tests somehow and maybe give me false positive and/or false negatives results after running my tests.

Questions:

  1. Will each test case get a new ClassLoader so that all static references from previous runs are not accessible/visible anymore?

  2. Are there other things which lead to test dependency/coupling of test methods which may change the behavior (beside native library loading which I am not using)

  3. What about garbage collection/finalization, are they run after each test? (I don't rely on them, but I just want to get a complete picture)


UPDATE

According to the current answers it seems that junit is always sharing a single classloader between all test cases per vm/fork when using forkMode. (so forkMode="once" indeed means there's one classloader for all tests)

This has many advantages (faster tests and may cause tests to fail because of static coupling) but also some disadvantages (static coupling which will only work if a shared classloader is used -> false positive)

MRalwasser
  • 15,605
  • 15
  • 101
  • 147
  • the update seems like a new question and makes the existing answers seem out of place – oers Jun 09 '11 at 09:06
  • good idea. new questions can be found here: http://stackoverflow.com/questions/6321473/junit-how-to-avoid-false-positives-when-using-forkmode-once – MRalwasser Jun 12 '11 at 10:32
  • Your update a little misleading. JUnit is sharing a classloader between all test cases when using `forkMode="once", but not the other forkModess, as you can't share classloaders between running VMs – Duncan McGregor Jun 12 '11 at 13:02
  • i've updated the question - I hope that it's better now. – MRalwasser Jun 12 '11 at 14:18

3 Answers3

11
  1. The test runner will effectively make a single Suite of all of your tests and run them - so that only one classloader is involved.
  2. Yes that means that static data will be shared between tests, which can occasionally be handy, but will force you to cut down on the static coupling between clauses, which is a good thing.
  3. There isn't generally any explicit GC, but you can do your own.

Generally running all your tests in one VM is a good thing. It forces you to look at static coupling and is a lot quicker. Crucially, it's also the way that your IDE will be running them, and that really is the way that tests should be run - as close as possible to as often as you compile.

Duncan McGregor
  • 17,665
  • 12
  • 64
  • 118
  • 1
    I mostly agree with your statement, however shared static data may result in false positive results which creates the illusion that everything's alright (due to static coupling which will not be discovered in this case) – MRalwasser Jun 08 '11 at 15:26
  • That's true of course - I often end up finding those when a test fails when run in isolation. There's a case for randomizing the order of the tests I suppose. – Duncan McGregor Jun 08 '11 at 16:51
5

Note that the default mode forks a new VM for each test case (i.e. class) not for each test (i.e. method). In the application I am currently testing there are problems that arise when I reuse a VM for more than one test: objects and state are left over from earlier tests and stop later ones from working. This may not be a problem if your application is well structured and your tests are strictly self-contained. I doubt garbage collection runs automatically after each test: it is notoriously hard to ensure that it is called at any given time in any case.

Ben
  • 2,348
  • 2
  • 19
  • 28
  • Your statement about the default forkMode is not true according to http://ant.apache.org/manual/Tasks/junit.html – MRalwasser Jun 07 '11 at 12:03
  • 1
    Actually that page does say '"perTest" creates a new VM for each TestCase class.' which accords with my experience. However, it does use 'test' to mean 'test case' elsewhere. It seems the JUnit developers have not always been consistent in their terminology. – Ben Jun 07 '11 at 14:05
  • Ah, I see, thanks for pointing this out, I'll update the question. – MRalwasser Jun 08 '11 at 07:00
0

Looking at Stefan's blog entry about this I would venture to guess:

  1. you will only get a single class loader for forkMode="once"
  2. you won't have access to the Ant environment anymore
  3. the GC will be performed inside the spawned GC (if forceMode="once") and this means it's NOT after each test is run.
Jesse Webb
  • 43,135
  • 27
  • 106
  • 143
Liv
  • 6,006
  • 1
  • 22
  • 29
  • 1. From which part of the entry do you conclude that there's only a _one_ classloader shared for _all_ tests? 2. ant environment is out of scope (perTest is also forked). 3. System.gc() may be invoked implicitly... – MRalwasser Jun 06 '11 at 20:20