JUnit 4 : Expect one exception multiple times

Question

I have the following piece of code:

@Test(expected = IllegalArgumentException.class)
public void failureTest()    {
    testedObject.supposedToFail("");
    testedObject.supposedToFail(null);
}

When running this, I have no guarantee that I will throw an IllegalArgumentException with a null parameter. Indeed, whenever JUnit meets the first exception, it stops the run of the whole method.

Considering the number of test cases I have to try for this class (~20), I doubt writing 20 methods each expecting a specific exception (which are often the same, tho) would be efficient.

Is there any way to try for every method throwing a specific exception at once ? (e.g. with my sample, you would go through both methods)

Thank you

*"When running this, I have no guarantee that I will throw an `IllegalArgumentException` with a `null` parameter. "* why not? In a unit test the tested code is supposed to behave in exactly one way according to the inputs and the setup. — Timothy Truckle, Jan 03 '18 at 11:26
Use a good assertion library (or rewrite similar utility methods). For example: http://joel-costigliola.github.io/assertj/core-8/api/org/assertj/core/api/Assertions.html#assertThatThrownBy-org.assertj.core.api.ThrowableAssert.ThrowingCallable-, http://joel-costigliola.github.io/assertj/core-8/api/org/assertj/core/api/Assertions.html#assertThatExceptionOfType-java.lang.Class- — JB Nizet, Jan 03 '18 at 11:31
@Timothy because the @Expected annotation works this way. When it catches the first `IllegalException` it considers the whole test method as complete and skips everything else. — Yassine Badache, Jan 03 '18 at 11:55
@YassineBadache How can your *production code* throw multiple exeptions* in one invocation? — Timothy Truckle, Jan 03 '18 at 12:51
I think you did not get the goal of this test. Take a code that throws an exception if your String parameter is `null` or empty. You get to test if either your parameter is null, or empty. That makes two cases to try. If your class contains 10 methods that expects this behavior, should I write 20 unit tests ? Goal here is to try all those methods throwing this exception, within a single test method. The below answer seems satisfactory, I will give it a shot. — Yassine Badache, Jan 03 '18 at 12:58
@YassineBadache What purpose does such test have? A unittest method should verify *a single assumption* about the production codes behavior. The reason is that in this case your test name (if carefully choosen) already tessl you what went wrong without having to debug. So yes, I did not get the goal of your test but you did not get the goal of unittesting... (refer here https://www.amazon.de/Art-Unit-Testing-Roy-Osherove) — Timothy Truckle, Jan 03 '18 at 13:03
Well, I guess making sure that a test fails accordingly to a said parameter is not the right way to do so. Thank you for your contribution. — Yassine Badache, Jan 03 '18 at 13:07

Bentaye · Accepted Answer · 2018-09-25T08:21:32.110

I would use an auxiliary method and do it this way:

EDIT: This does not work, see alternative working solution

@Rule
public ExpectedException thrown = ExpectedException.none();

@Test
public void myTest() {
    List<Object[]> paramsList = Arrays.asList(
        new Object[] {"", IllegalArgumentException.class},
        new Object[] {null, NullPointerException.class});
    paramsList.forEach(a -> assertExceptionForParam((String)a[0], (Class)a[1]));
}

private void assertExceptionForParam(String param, Class expectedExceptionClass) {
    thrown.expect(expectedExceptionClass);
    testedObject.supposedToFail(param);
}

ALTERNATIVE WORKING SOLUTION, CHANGE AFTER COMMENT BY MIRZAK

My solution seems to actually only test the first case in the list. Here is a working version that will test them all

@Test
public void myTest() {
    List<Object[]> paramsList = Arrays.asList(
        new Object[] {null, NullPointerException.class},
        new Object[] {"", IllegalArgumentException.class},
        new Object[] {"zip", NullPointerException.class});
    paramsList.forEach(a -> assertExceptionForParam((String)a[0], (Class)a[1]));
}

private void assertExceptionForParam(String param, Class expectedExceptionClass) {
    boolean pass = false;
    try {
        testedObject.supposedToFail(param);
    } catch(Exception e) {
        pass = e.getClass() == expectedExceptionClass;
    }
    Assert.assertTrue("test failed for param:" + param + " and Exception "+expectedExceptionClass, pass);
}

This outputs, as expected:

java.lang.AssertionError: test failed for param:zip and Exception class java.lang.NullPointerException

If you add a value to the list (at the end) for which an exception should not be thrown this test will still pass. Is there a way to "reset" thrown object ? — mirzak, Sep 25 '18 at 06:36
@mirzak I am afraid you are right. I changed the post with a working solution that I tested — Bentaye, Sep 25 '18 at 08:21

score 1 · Answer 2 · answered Aug 09 '18 at 11:53

Unfortunately, auxiliary method won't work in that case, because @Test method will pass even if only one of auxiliary method invocations throws exception.

I didn't find a beautiful way to implement such test and end up with something like this:

@Rule
public ExpectedException thrown = ExpectedException.none();

@Test
public void testThrowIllegalArgumentExceptionForEmptyParam() {
    assertExceptionForParam("", IllegalArgumentException.class);
}

@Test
public void testThrowNullPointerExceptionForNullParam() {
    assertExceptionForParam(null, NullPointerException.class);
}

private void assertExceptionForParam(String param, Class expectedExceptionClass) {
    thrown.expect(expectedExceptionClass);
    testedObject.supposedToFail(param);
}

In case of test failure you will have clear understanding where and when it failed and also your code won't contain a lot of boilerplate.

score -1 · Answer 3 · answered Jan 04 '18 at 01:13

Note, this is not a solution for junit but let me share how would be implemented the same test on spock:

def "supposedToFail(String) throws IllegalArgumentException when the given argument is null or blank"() {
    when:
    testedObject.supposedToFail(givenArgument)

    then:
    thrown(IllegalArgumentException)

    where:
    givenArgument << [null, '', '       ', '\n']
}

Spock can be used on your Java project together with junit without any problems.

JUnit 4 : Expect one exception multiple times

3 Answers3