Writing first JUnit test

Question

So I've read the official JUnit docs, which contain a plethora of examples, but (as with many things) I have Eclipse fired up and I am writing my first JUnit test, and I'm choking on some basic design/conceptual issues.

So if my WidgetUnitTest is testing a target called Widget, I assume I'll need to create a fair number of Widgets to use throughout the test methods. Should I be constructing these Widgets in the WidgetUnitTest constructor, or in the setUp() method? Should there be a 1:1 ratio of Widgets to test methods, or do best practices dictate reusing Widgets as much as possible?

Finally, how much granularity should exist between asserts/fails and test methods? A purist might argue that 1-and-only-1 assertions should exist inside a test method, however under that paradigm, if Widget has a getter called getBuzz(), I'll end up with 20 different test methods for getBuzz() with names like

@Test
public void testGetBuzzWhenFooIsNullAndFizzIsNonNegative() { ... }

As opposed to 1 method that tests a multitude of scenarios and hosts a multitude of assertions:

@Test
public void testGetBuzz() { ... }

Thanks for any insight from some JUnit maestros!

Tomasz Nurkiewicz · Accepted Answer · 2011-10-05T18:28:19.400

Pattern

Interesting question. First of all - my ultimate test pattern configured in IDE:

@Test
public void shouldDoSomethingWhenSomeEventOccurs() throws Exception
{
    //given

    //when

    //then
}

I am always starting with this code (smart people call it BDD).

In given I place test setup unique for each test.
when is ideally a single line - the thing you are testing.
then should contain assertions.

I am not a single assertion advocate, however you should test only single aspect of a behavior. For instance if the the method should return something and also has some side effects, create two tests with same given and when sections.

Also the test pattern includes throws Exception. This is to handle annoying checked exceptions in Java. If you test some code that throws them, you won't be bothered by the compiler. Of course if the test throws an exception it fails.

Setup

Test setup is very important. On one hand it is reasonable to extract common code and place it in setup()/@Before method. However note that when reading a test (and readability is the biggest value in unit testing!) it is easy to miss setup code hanging somewhere at the beginning of the test case. So relevant test setup (for instance you can create widget in different ways) should go to test method, but infrastructure (setting up common mocks, starting embedded test database, etc.) should be extracted. Once again to improve readability.

Also are you aware that JUnit creates new instance of test case class per each test? So even if you create your CUT (class under test) in the constructor, the constructor is called before each test. Kind of annoying.

Granularity

First name your test and think what use-case or functionality you want to test, never think in terms of:

this is a Foo class having bar() and buzz() methods so I create FooTest with testBar() and testBuzz(). Oh dear, I need to test two execution paths throughout bar() - so let us create testBar1() and testBar2().

shouldTurnOffEngineWhenOutOfFuel() is good, testEngine17() is bad.

More on naming

What does the testGetBuzzWhenFooIsNullAndFizzIsNonNegative name tell about the test? I know it tests something, but why? And don't you think the details are too intimate? How about:

@Test shouldReturnDisabledBuzzWhenFooNotProvidedAndFizzNotNegative`

It both describes the input in a meaningful manner and your intent (assuming disabled buzz is some sort of buzz status/type). Also note we no longer hardcode getBuzz() method name and null contract for Foo (instead we say: when Foo is not provided). What if you replace null with null object pattern in the future?

Also don't be afraid of 20 different test methods for getBuzz(). Instead think of 20 different use cases you are testing. However if your test case class grows too big (since it is typically much larger than tested class), extract into several test cases. Once again: FooHappyPathTest, FooBogusInput and FooCornerCases are good, Foo1Test and Foo2Test are bad.

Readability

Strive for short and descriptive names. Few lines in given and few in then. That's it. Create builders and internal DSLs, extract methods, write custom matchers and assertions. The test should be even more readable than production code. Don't over-mock.

I find it useful to first write a series of empty well-named test case methods. Then I go back to the first one. If I still understand what was I suppose to test under what conditions, I implement the test building a class API in the meantime. Then I implement that API. Smart people call it TDD (see below).

Recommended reading:

What a great answer! I second everything in it. Next step is to talk about mocking :) — Guillaume, Oct 05 '11 at 18:06
@Tomasz - you are the new champion of SO. This is the most ridiculously-well answered question I've ever received here. Awesome, awesome answer. Thank you. — IAmYourFaja, Oct 05 '11 at 18:09
@Tomasz - why does your pattern method throw an Exception? Isn't this handled in the assertions, or am I missing something here? — IAmYourFaja, Oct 05 '11 at 18:15
@AdamTannon: I updated my answer to address your concerns about `Exception`. — Tomasz Nurkiewicz, Oct 05 '11 at 18:21

score 1 · Answer 2 · answered Oct 05 '11 at 17:52

1

You would create a new instance of the class under test in your setup method. You want each test to be able to execute independently without having to worry about any unwanted state in the object under test from another previous test.

I would recommend having separate test for each scenario/behavior/logic flow that you need to test, not one massive test for everything in getBuzz(). You want each test to have a focused purpose of what you want to verify in getBuzz().

answered Oct 05 '11 at 17:52

Kristian

6,443
6
27
29

Thanks for the response Kristian - but is it typical then for test methods to take on extremely long, cumbersome names like the one in the example I provided? – IAmYourFaja Oct 05 '11 at 17:53
Long names yes, maybe not exactly as your example. We try to name them as to what they are testing, testGetBuzzWhenNoNetworkConnection() – Kristian Oct 05 '11 at 18:06
1

One of the main points of testing (not the most important, but close) is to serve as a documentation to your class: don't worry about cumbersome name, the most important is they say exactly what they test – Guillaume Oct 05 '11 at 18:25

score 1 · Answer 3 · answered Oct 05 '11 at 18:21

Rather than testing methods try to focus on testing behaviors. Ask the question "what should a widget do?" Then write a test that affirms the answer. Eg. "A widget should fidget"

public void setUp() throws Exception {
   myWidget = new Widget();
}

public void testAWidgetShouldFidget() throws Exception {
  myWidget.fidget();
}

compile, see "no method fidget defined " errors, fix the errors, recompile the test and repeat. Next ask the question what should the result of each behavior be, in our case what happens as the result of fidget? Maybe there is some observable output like a new 2D coordinate position. In this case our widget would be assumed to be in a given position and when it fidgets it's position is altered some way.

public void setUp() throws Exception {
   //Given a widget
   myWidget = new Widget();
   //And it's original position
   Point initialWidgetPosition = widget.position();
}


public void testAWidgetShouldFidget() throws Exception {
  myWidget.fidget();
}

public void testAWidgetPositionShouldChangeWhenItFidgets() throws Exception {
  myWidget.fidget();
  assertNotEquals(initialWidgetPosition, widget.position());
}

Some would argue against both tests exercising the same fidget behavior but it makes sense to single out the behavior of fidget independent of how it impacts widget.position(). If one behavior breaks the single test will pinpoint the cause of failure. Also it is important to state that the behavior can be exercised on its own as a fulfillment of the spec (you do have program specs don't you?) that suggests you need a fidgety widget. In the end it's all about realizing your program specs as code that exercises your interfaces which demonstrate both that you've completed the spec and secondly how one interacts with your product. This is in essence how TDD should work. Any attempt to resolve bugs or test the product usually results in a frustrating pointless debate over which framework to use, level of coverage and how fine grained your suite should be. Each test case should be an exercise of breaking down your spec into a component where you can begin phrasing with Given/When/Then. Given {some application state or precondition} When {a behavior is invoked} Then {assert some observable output}.

score 0 · Answer 4 · edited Nov 07 '13 at 16:45

First of all, the setUp and the tearDown Methods will be called before and after each Test, so the setUp Method should create the objects, if you need them in every test, and test-specific things may be done in the test itself.

Second, it is up to you how you want to test your program. Obviously you could write a test for every possible situation in your program and end up with a gazillion tests for every method. Or you could write just one test for every method, which checks every possible scenario. I would recommend a mixture between both ways. You really don't need test for trivial getters/setters, but writing just one test for a method may result in confusion if the test fails. You should decide, which Methods are worth testing, and which scenarios are worth testing. But in principle every scenario should have its own Test.

Mostly I end up with a code coverage of 80 to 90 percent with my tests.

The @Test annotation indicates he's using JUnit 4. So setUp and tearDown will NOT be called, unless they're annotated with "Before" and "After" (and they can be named anything, although setUp and tearDown are standard names) — Guillaume, Oct 05 '11 at 18:04

Guillaume · Answer 5 · 2011-10-05T18:22:39.987

I completely second Tomasz Nurkiewicz answer, so I'll say that rather than repeating everything he said.

A couple more points:

Don't forget to test error cases. You can consider something like that:

@Test
public void throwExceptionWhenConditionOneExist() {
    // setup
    // ...
    try {
       classUnderTest.doSomething(conditionOne);
       Assert.fail("should have thrown exception");
    } catch (IllegalArgumentException expected) {
       Assert.assertEquals("this is the expected error message", expected.getMessage());
    } 
}

Also, it has GREAT value to start writing your tests before even thinking about the design of your class under test. If you're a beginner on unit-testing, I cannot emphasize enough learning this technique at the same time (this is called TDD, test-driven development) which proceeds like that:

You think about what user case you have for your user requirements
You write a basic first test for it
You make it compile (by creating needed classes -including your class under test-, etc.)
You run it: it should fail
Now you implement the functionality of the class under test that will make it pass (and nothing more)
Rinse, and repeat with a new requirement

When all your requirements have passing tests, then you're done. You NEVER write anything in your production code that doesn't have a test before (exceptions to that is logging code, and not much more).

TDD is invaluable in producing good quality code, not over-engineering requirements, and making sure you have a 100% functional coverage (rather than line coverage, which is usually meaningless). It requires a change in the way you consider coding, that's why it's valuable to learn the technique at the same time as testing. Once you get it, it will become natural.

Next step is looking into Mocking strategies :)

Have fun testing.