0

I'm looking to distribute tests to multiple instances of the junit5 standalone console whereby each instance reads off of a queue. Each instance of the runner would use the same test.jar on the classpath, so I'm not trying to distribute the byte code of the actual tests here, just the names of the tests / filter pattern strings.

From the junit 5 advanced topics doc, I think the appropriate place to extend junit 5 to do this is using the platform launcher api. I cobbled this snippet together largely with the sample code in the guide. I think this is what I need to write but I'm concerned I'm oversimplifying the effort involved here:

// keep pulling test classes off the queue until its empty
while(myTestQueue.isNotEmpty()) {
    String classFromQueue = myTestQueue.next(); //returns "org.myorg.foo.fooTests"
    LauncherDiscoveryRequest request = LauncherDiscoveryRequestBuilder.request()
        .selectors(selectClass(classFromQueue)).build();
        
    SummaryGeneratingListener listener = new SummaryGeneratingListener();
    try (LauncherSession session = LauncherFactory.openSession()) {
        Launcher launcher = session.getLauncher();
        launcher.registerTestExecutionListeners(listener);
        TestPlan testPlan = launcher.discover(request);
        launcher.execute(testPlan);
    }   
    TestExecutionSummary summary = listener.getSummary();
    addSummary(summary);
}

Questions:

  1. Will repeatedly discovering and executing in a while loop violate the normal test lifecycle? I'm a little fuzzy on whether discovery is a one time thing that's supposed to happen before all executions.
  2. If I assume that it's ok to repeatedly discover then execute, I see the HierarchicalTestEngine may be an even better place to read from a queue since that seems to be used for implementing parallel execution. Is this more suitable for my use case? Would the implementation be essentially the same as what I have above except maybe I wouldn't need to handle accumulating test summaries?

Approaches I do not want to take: I am not looking to use the new features of junit 5 aimed at parallelizing test execution within the same jvm. I'm also not looking to divide the tests or classes up ahead of time; starting each console runnner instance with a pre-determined subset of tests.

Damon
  • 305
  • 1
  • 5
  • 13
  • You’re code looks fine at first glance. Does it work? As for HierarchicalTestEngine, it’s a superclass for engines that aim at handling test and test container hierarchies just like Jupiter. So it does not look like a way to go unless you want to wrap the queuing in an engine of its own. And I don’t see why you would. – johanneslink Mar 29 '22 at 05:43
  • I'm actually encouraged to hear that discover and execute in that block can be repeated. I honestly thought I was oversimplifying the effort involved and so I hadn't yet setup a project to test this out with a list of test classes. I often see examples of distributed testing achieved by pre-allocating tests to separate runner instances and so I assumed that extending junit5 must be harder than it looked. Stay tuned and thank you for steering me towards the above approach. Hoping I can provide working code that can be used in an accepted answer. – Damon Mar 30 '22 at 01:45
  • 1
    It worked!!!! I'm a blown away at how nicely this works. I need a bit more time to think through what I'm seeing with handling the results. I left that part out for now but I just thought I'd update you. I still plan to post an answer with my code. – Damon Mar 31 '22 at 14:32
  • Just posted my code @johanneslink. Figured I'd wait a bit and see if you wanted to weigh in before I mark my own answer as accepted. – Damon Apr 04 '22 at 17:58

1 Answers1

1

Short Answer

The code posted in the question (loosely) illustrates a valid approach. There is no need to create a custom engine. Leveraging the platform launcher api to repeatedly discover and execute tests does work. I think it's worth highlighting that you do not have to extend junit5 This isn't executed through an extension that you need to register as I'd originally assumed. You're just simply leveraging the platform launcher api to discover and execute tests.

Long Answer

Here is some sample code with a simple queue of tests class names that exist on the class path. While the queue is not empty, an instance of the testNode class will discover and execute each of the three test classes and write a LegacyXmlReport.

TestNode Code:

package org.sample.node;
import org.junit.platform.launcher.Launcher;
import org.junit.platform.launcher.LauncherDiscoveryRequest;
import org.junit.platform.launcher.LauncherSession;
import org.junit.platform.launcher.TestPlan;
import org.junit.platform.launcher.core.LauncherConfig;
import org.junit.platform.launcher.core.LauncherDiscoveryRequestBuilder;
import org.junit.platform.launcher.core.LauncherFactory;
import org.junit.platform.launcher.listeners.SummaryGeneratingListener;
import org.junit.platform.reporting.legacy.xml.LegacyXmlReportGeneratingListener;

import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.PrintWriter;
import java.nio.file.Paths;
import java.util.LinkedList;
import java.util.Queue;

import static org.junit.platform.engine.discovery.DiscoverySelectors.selectClass;
public class TestNode {

    public void run() throws FileNotFoundException {

        // keep pulling test classes off the queue until its empty
        Queue<String> queue = getQueue();
        while(!queue.isEmpty()) {
            String testClass = queue.poll();
            LauncherDiscoveryRequest request = LauncherDiscoveryRequestBuilder.request()
                    .selectors(selectClass(testClass)).build();


            LauncherConfig launcherConfig = LauncherConfig.builder()
                    .addTestExecutionListeners(new LegacyXmlReportGeneratingListener(Paths.get("target"), new PrintWriter(new FileOutputStream("log.txt"))))
                    .build();

            SummaryGeneratingListener listener = new SummaryGeneratingListener();
            try (LauncherSession session = LauncherFactory.openSession(launcherConfig)) {
                Launcher launcher = session.getLauncher();
                launcher.registerTestExecutionListeners(listener);
                TestPlan testPlan = launcher.discover(request);
                launcher.execute(testPlan);
            }
        }
    }

    private Queue<String> getQueue(){
        Queue<String> queue = new LinkedList<>();
        queue.add("org.sample.tests.Class1");
        queue.add("org.sample.tests.Class2");
        queue.add("org.sample.tests.Class3");
        return queue;
    }

    public static void main(String[] args) throws FileNotFoundException {
        TestNode node = new TestNode();
        node.run();
    }
}

Tests executed by TestNode

I'm just showing 1 of the three test classes since they're all the same thing with different class names. They reside in src/main/java and NOT src/test/java. This is an admittedly weird yet common pattern in maven for packaging tests into a fat jar.

package org.sample.tests;

import org.junit.jupiter.api.Test;

public class Class1 {
    @Test
    void test1() {
        System.out.println("Class1 Test 1");
    }

    @Test
    void test2() {
        System.out.println("Class1 Test 2");
    }

    @Test
    void test3() {
        System.out.println("Class1 Test 3");
    }
}
Damon
  • 305
  • 1
  • 5
  • 13