why this is not working - Unit testing a synchronous method using parameterized data of Junit?

Question

I am trying to learn the JUnit and wanted to extend it to test in a multi-threaded way.

The class I want to test is PrimeNumberValidator. This just tests if a passed in number is prime or not.

package com;

public class PrimeNumberValidator {
    public Boolean validate(final Integer primeNumber) {
        System.out.println("Validating .............:" + primeNumber);
        for (int i = 2; i < (primeNumber / 2); i++) {
            if (primeNumber % i == 0) {
                return false;
            }
        }

        return true;
    }
}

The PrimeNumberValidatorTest is the test class. The 2 of the test data are wrong and I have done this purposely to test for failure.
The test method testPrimeNumberValidator runs perfectly fine. But, the multi-threaded version
testMultiThreadedPrimeNumberValidator always says 'pass' for even wrong data.
Why is this happening like this?
How do solve this ?

package com;

import static org.junit.Assert.assertEquals;

import java.util.Arrays;
import java.util.Collection;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;

@RunWith(Parameterized.class)
public class PrimeNumberValidatorTest {
    private Integer primeNumber;
    private Boolean expectedValidation;
    private PrimeNumberValidator primeNumberValidator;

    @Before
    public void initialize() {
    primeNumberValidator = new PrimeNumberValidator();
    }

    // Each parameter should be placed as an argument here
    // Every time runner triggers, it will pass the arguments from parameters we defined
    public PrimeNumberValidatorTest(Integer primeNumber, Boolean expectedValidation) {
    this.primeNumber = primeNumber;
    this.expectedValidation = expectedValidation;
    }

    @Parameterized.Parameters
    public static Collection primeNumbers() {
    return Arrays.asList(new Object[][] {
        { 2, Boolean.FALSE},// 2 is prime so Test should fail
        { 6, Boolean.FALSE}, //is NOT prime so test should pass
        { 19, Boolean.TRUE},//is prime so test should pass
        { 22, Boolean.TRUE} //is NOT prime so test should fail
    });
    }

    // This test will run 4 times since we have 4 parameters defined
    @Test
    public void testPrimeNumberValidator() {
    assertEquals(expectedValidation, primeNumberValidator.validate(primeNumber));
    }

    @Test
    public void testMultiThreadedPrimeNumberValidator() {

    ExecutorService executor = Executors.newFixedThreadPool(100);
    executor.submit(new Runnable() {

        public void run() {
        for (int i = 0; i < 100; i++) {
            assertEquals(expectedValidation, primeNumberValidator.validate(primeNumber));
        }   
        }
    }); 
    }
}

Referring to
http://www.youtube.com/watch?v=wDN_EYUvUq0 as mentioned in one of the posts
Weird problem using JUnit in multi-thread environment, tried as below.
The exceptions are thrown, but the JUnit does not report failure :(

package com;

import static org.junit.Assert.assertEquals;

import java.util.Arrays;
import java.util.Collection;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

import org.junit.After;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;

@RunWith(Parameterized.class)
public class PrimeNumberValidatorTest {
    volatile Exception exception;
    volatile Error error;

    private Integer primeNumber;
    private Boolean expectedValidation;
    private PrimeNumberValidator primeNumberValidator;

    @Before
    public void initialize() {
    primeNumberValidator = new PrimeNumberValidator();
    }

    // Each parameter should be placed as an argument here
    // Every time runner triggers, it will pass the arguments from parameters we defined
    public PrimeNumberValidatorTest(Integer primeNumber, Boolean expectedValidation) {
    this.primeNumber = primeNumber;
    this.expectedValidation = expectedValidation;
    }

    @Parameterized.Parameters
    public static Collection primeNumbers() {
    return Arrays.asList(new Object[][] {
        { 2, Boolean.FALSE},// 2 is prime so Test should fail
        { 6, Boolean.FALSE}, //is NOT prime so test should pass
        { 19, Boolean.TRUE},//is prime so test should pass
        { 22, Boolean.TRUE} //is NOT prime so test should fail
    });
    }

    // This test will run 4 times since we have 4 parameters defined
    @Test
    @Ignore
    public void testPrimeNumberValidator() {
    assertEquals(expectedValidation, primeNumberValidator.validate(primeNumber));
    }

    @Test
    public void testMultiThreadedPrimeNumberValidator() {

    ExecutorService executor = Executors.newFixedThreadPool(100);
    executor.submit(new Runnable() {

        public void run() {
        for (int i = 0; i < 1; i++) {
            try{
                assertEquals(expectedValidation, primeNumberValidator.validate(primeNumber));
            }catch(Error e){
                System.out.println("error thrown :" + e);
                error =e;
            }catch(Exception e){
                exception=e;
                System.out.println("exception thrown :" + e);
            }
        }   
        }
    }); 
    }

    @After
    public void runAfterEveryTest() throws Exception{
    if(null != error){
        System.out.println("runAfterEveryTest throwing error...............");
        throw error;
    }
    if(null != exception){
        System.out.println("runAfterEveryTest throwing exception...............");
        throw exception;
    }
    }
}

John B · Answer 1 · 2013-09-19T11:12:32.397

The reason the multi-threaded is always passing is the following:

first, the test method does nothing to wait for the multiple threads to complete therefore the @Test method is exiting before any errors are reported
second, the way that testing works is that the assert method throws an exception. The JUnit framework catches this exception by wrapping the @Test method and fails the test if an unexpected exception is thrown. However, in the multi-threaded test no exception is thrown from the @Test method because they are thrown within the threads and the test method does nothing to check this. One solution for this part would be to use an ErrorCollector

score 0 · Answer 2 · answered Sep 19 '13 at 17:40

It got resolved with the below code. The addtion of Future related code made it work.

@Test
public void testMultiThreadedPrimeNumberValidator() throws InterruptedException, ExecutionException {

    ExecutorService executor = Executors.newFixedThreadPool(100);
    Future future = executor.submit(new Runnable() {

        public void run() {
            for (int i = 0; i < 10; i++) {
                try{
                    assertEquals(expectedValidation, primeNumberValidator.validate(primeNumber));
                }catch(Error e){
                    System.out.println("error thrown :" + e);
                    //error =e;
                    System.out.println("error set to :" + e);
                }catch(Exception e){
                    System.out.println("exception thrown :" + e);
                    //exception=e;
                    System.out.println("Exception set to :" + e);
                }
            }   
        }
    }); 
    future.get();
}

why this is not working - Unit testing a synchronous method using parameterized data of Junit?

2 Answers2