JUnit5: Before and After Suite method invocation

Question

I've spent a few days trying to find out a solution, so I know all the basic answers

I've read the documentation, and I know, that @BeforeClass/@AfterClass is replaced with @BeforeAll/@AfterAll, @RunWith no longer exists; superseded by @ExtendWith I've read all the topics here on the stackoverflow (removed links, since stackoverflow pretends it's a spam:(

I used to run some JUnit4 tests in the suites. Each Suite class Suite1.class, Suite2.class had several @Test methods and @BeforeClass/@AfterClass were running exactly before/after all the testing methods.

@RunWith(StopOnFailureSuite.class)
@Suite.SuiteClasses({
        Test1.class,
        Test2.class
})

public class TSuite_20 {
    private static final byte SUITE_NUMBER = 20;

    @BeforeClass
    public static void setUpClass() {
        //some logic for suite setup
    }

    @AfterClass
    public static void tearDownClass() {
        //some logic for teardown
    }
}

As I wanted to use @ParameterizedTests I need to migrate to JUnit5. And suddenly I realised, that exact the same behaviour, that used to be in JUnit4 is no more achievable. [run some custom setup code; run several test classes, which may contain several @Test methods; run some custom tear down code];

Does anybody know (better with examples) an approach to make it with JUnit 5?

Option 1 This code will never execute BeforeAfterSuite#beforeAll and BeforeAfterSuite#afterAll

@ExtendWith(BeforeAfterSuite.class)
@Suite
@SelectClasses({
        Test1.class,
        Test2.class
})
public class TSuite_20 {
    public static final byte SUITE_NUMBER = 20;
}

public class BeforeAfterSuite implements BeforeAllCallback, AfterAllCallback
/*,TestInstancePreConstructCallback, BeforeTestExecutionCallback, 
AfterTestExecutionCallback, ExtensionContext.Store.CloseableResource*/ {
    private static boolean started = false;

    @Override
    public void beforeAll(ExtensionContext context) {
        if (!started) {
            started = true;
            //before stuff
        }
    }

    @Override
    public void afterAll(ExtensionContext context) throws Exception {
        //after all;
    }
}

Option 2 I was just curious, how will JUnit treat suite class if I put a test method into it... This code will execute BeforeAfterSuite#beforeAll and BeforeAfterSuite#afterAll once, before and after TSuite_20#test

@ExtendWith(BeforeAfterSuite.class)
@Suite
@SelectClasses({
        Test1.class,
        Test2.class
})
public class TSuite_20 {
    public static final byte SUITE_NUMBER = 20;
    
    @Test
    public void test() {
    }
}

Option 3 We also can apply @ExtendWith(BeforeAfterSuite.class) per Test class which will results in a BeforeAfterSuite#beforeAll and BeforeAfterSuite#afterAll per Test class. (in this example - 2 times).

@ExtendWith(BeforeAfterSuite.class)
public class Test1 {
    
    @Test
    public void test11() {
    }
    @Test
    public void test12() {
    }
}

@ExtendWith(BeforeAfterSuite.class)
public class Test2 {
    
    @Test
    public void test21() {
    }
    @Test
    public void test22() {
    }
}

Option 4 I also give a shot for

• a Suite class without @ExtendWith() and @BeforeAll + @AfterAll; (as expected nothing happened)`
• a Suite class without @ExtendWith() and @BeforeAll + @Test + @AfterAll; (as expected single execution of BeforeAll/AfterAll for the specific Suite class)`

Option 5 Listeners were my last hope to achieve the desired behaviour. I've created my own impl for LauncherSessionListener, just because I've thought it will allow me to execute smth exactly before tests start.

public class BeforeAfterSuiteLauncher implements LauncherSessionListener {
    private static boolean started = false;
    @Override
    public void launcherSessionOpened(LauncherSession session) {
        if (!started) {
            started = true;
            //before all
        }
    }
    @Override
    public void launcherSessionClosed(LauncherSession session) {
        //after all
    }
}

And I've added also some default impl CompositeLauncherSessionListener

Packages structure screenshot to show Java SPI configuration: LauncherSessionListener

For TestExecutionListener I've added two default impls, just to catch at least one Listener:

org.junit.platform.launcher.listeners.LoggingListener
org.junit.platform.launcher.listeners.SummaryGeneratingListener

and one custom

public class BeforeAfterExecutionListener implements TestExecutionListener {
    @Override
    public void testPlanExecutionStarted(TestPlan testPlan) {
        //before all
    }

    @Override
    public void testPlanExecutionFinished(TestPlan testPlan) {
        //after all
    }
}

Packages structure screenshot to show Java SPI configuration:TestExecutionListener

And only SummaryGeneratingListener was triggered!

What am I doing wrong? Why my BeforeAfterExecutionListener impl was not loaded and triggered?

P.S. All of the above code was executed under Intellij Idea 2021.1.3 Ultimate Edition

java version "1.8.0_341"
Java(TM) SE Runtime Environment (build 1.8.0_341-b10)
Java HotSpot(TM) 64-Bit Server VM (build 25.341-b10, mixed mode)

here is intelliJs command:

C:\Tools\jdk\bin\java.exe -agentlib:jdwp=transport=dt_socket,address=127.0.0.1:61280,suspend=y,server=n -ea -Didea.test.cyclic.buffer.size=1048576 -javaagent:C:\Users\userName\AppData\Local\JetBrains\IntelliJIdea2021.1\groovyHotSwap\gragent.jar -javaagent:C:\Users\userName\AppData\Local\JetBrains\IntelliJIdea2021.1\captureAgent\debugger-agent.jar -Dfile.encoding=UTF-8 -classpath C:\Users\userName\AppData\Local\Temp\classpath1705687115.jar com.intellij.rt.junit.JUnitStarter -ideVersion5 -junit5 com.testdirectly.application.testcase.TSuite_20

Which results in JUnit5IdeaTestRunner

My gradle dependencies

dependencies {
        //JUnit platform
        // to group tests by package, by class name, by class name pattern, etc (use @Suite, @SelectClasses) :junit-platform-suite-api:1.9.2
        // and to filter/discover and run them         (SuiteLauncher, SuiteTestEngine, SuiteTestDescriptor) :junit-suite-engine:1.9.2
        testImplementation "org.junit.platform:junit-platform-suite:1.9.2"
        //Launcher, engine discovery
        testImplementation "org.junit.platform:junit-platform-launcher:1.9.2"//to run tests

        //JUnit Jupiter
        //to use assertions and so on
        testImplementation "org.junit.jupiter:junit-jupiter-api:5.9.2"
        //to use @ParameterizedTest
        testImplementation "org.junit.jupiter:junit-jupiter-params:5.9.2"
        //Jupiter engine to run junit5 tests (JupiterTestEngine, Extensions, etc)
        testRuntimeOnly "org.junit.jupiter:junit-jupiter-engine:5.9.2"
}

Answer 1

Short answer

Option 5 works if you tweak it slightly (see below).

Long answer

Options 1 and 2 do not work because @ExtendWith is a Jupiter extension mechanism whereas @Suite triggers an engine of its own. Test engines do not combine on the JUnit platform.

The same goes for option 4 since @BeforeAll and @AfterAll are Jupiter annotations.

Making Option 5 Work

First of all I'd suggest to use TestExecutionListener instead of LauncherSessionListener because the latter is still experimental.

Thus we have

package my.project.suites;

import org.junit.platform.launcher.*;

public class BeforeAfterSuiteListener implements TestExecutionListener {

    @Override
    public void testPlanExecutionStarted(TestPlan testPlan) {
        System.out.println("before all");
    }

    @Override
    public void testPlanExecutionFinished(TestPlan testPlan) {
        System.out.println("after all");
    }
}

The missing thing is now that you'll have to register BeforeAfterSuiteListener globally. In classpath-based Java you do that through a resource file META-INF/services/org.junit.platform.launcher.TestExecutionListener:

my.project.suites.BeforeAfterSuiteListener

Now before all and after all should show up in your output exactly once per test run.

Answer 2

I had the same problem and I implemented it this way for the finalization of the Suite. Didn't try the init as I didn't need it, but it should work the same way. Not sure this code works for parallels tests though.

@SelectClasses({
      Test1.class,
      Test2.class,
      Test3.class
      MySuite.FinalizeSuite.class
    })
public class MySuite
{
    public static class FinalizeSuite
    {
        @Test
        public void nothing()
        {}
        @AfterAll
        public static void tearDown()
        {
            //my suite finalization code
        }       
    }
    
}

I don't really like it as it is a lot of code to replace the Junit4 @BeforeSuite / @AfterSuite, but it is the best compromise I found for a simple solution.