Erlang - Eunit setup function doesn't run

Question

I would like to write tests, where setup and cleanup functions don't execute for every single test, but once - at the start of testing and at the end of testing.

So i tried this way:

generator_test_() ->
{setup, fun setup/0, fun cleanup/1,
    {inorder,
        [
            some_test1()
            %...
        ]
    }
}.

Where some_test1/0 is

some_test1() ->
    begin
        Value1 = something1(),
        Value2 = something2(),
        [?_assertEqual(Value1, Value2)]
    end.

Setup doesn't execute this way.

But this way:

some_test1() ->
    ?_test(begin
        Value1 = something1(),
        Value2 = something2(),
        [?_assertEqual(Value1, Value2)]
    end).

or

some_test1() ->
    [?_assertEqual(something1(), something2())].

It's all good.

I don't understand why it works and my problem is I cannot write multiple asserts in ?_test(....).

some_test1() ->
    ?_test(begin
        Value1 = something1(),
        Value2 = something2(),
        [?_assertEqual(Value1, Value2),
        ?_assertEqual(Value1, Value2)]
    end).

By running eunit:test(my_module, [verbose]). I see that only 1 test has passed.

UPDATE

Example1

A small example:

-module(example).

-include_lib("eunit/include/eunit.hrl").

generator_test_() ->
    {setup, fun setup/0, fun cleanup/1,
        {inorder,
            fun() -> some_test1() end
        }
    }.

some_test1() ->
    begin
        X = 3,
        Y = 4,
        %%% This is a test set/deep list as it
        %%% contains two tests.
        [?_assertEqual(X, Y),
        ?_assertEqual(ok, ok)]
    end.

setup() ->
    erlang:display("---SETUP CALLED---"),
    "setup".

cleanup(_) ->
    ok.

Running eunit:

eunit:test(example, [verbose]).
======================== EUnit ========================
"---SETUP CALLED---"
module 'example'
  example: generator_test_...ok
  [done in 0.004 s]
=======================================================
  Test passed.
ok

A small modification:

some_test1() ->
    begin
        X = 3,
        Y = 4,
        %%% This is a test set/deep list as it
        %%% containst two tests.
        [?_assertEqual(X, Y),
        ?_assertEqual(it_doesnt_work, ok)]
    end.

And the test still passes.

Example2

However

some_test1() ->
    begin
        X = 3,
        Y = 4,
        %%% These are just two simple tests
        ?assertEqual(X, Y),
        ?assertEqual(it_doesnt_work, ok)
    end.

Fails

eunit:test(example, [verbose]).
======================== EUnit ========================
"---SETUP CALLED---"
module 'example'
  example: generator_test_...*failed*
in function example:'-some_test1/0-fun-0-'/1 (example.erl, line 17)
in call from example:some_test1/0 (example.erl, line 17)
**error:{assertEqual_failed,[{module,example},
                     {line,17},
                     {expression,"Y"},
                     {expected,3},
                     {value,4}]}


  [done in 0.003 s]
=======================================================
  Failed: 1.  Skipped: 0.  Passed: 0.
error

BUT

Example3

generator_test_() ->
    {setup, fun setup/0, fun cleanup/1,
        {inorder,
            some_test1()
        }
    }.

some_test1() ->
    [?_assertEqual(3, 3),
    ?_assertEqual(ok, ok)].

eunit:test(example, [verbose]).
======================== EUnit ========================
"---SETUP CALLED---"
module 'example'
  example:13: some_test1...ok
  example:14: some_test1...ok
  [done in 0.005 s]
=======================================================
  All 2 tests passed.
ok

Now I see that there were 2(!) tests.

I could just forget the block statement and write tests without variables, but I think that could be horrible to read and it should work with blocks.

Answer 1

The documentation says:

A simple test object is one of the following:

A nullary functional value (i.e., a fun that takes zero arguments). Examples:

   fun () -> ... end
   fun some_function/0
   fun some_module:some_function/0

A tuple {test, ModuleName, FunctionName}, where ModuleName and FunctionName are atoms, referring to the function

ModuleName:FunctionName/0

A pair {LineNumber, SimpleTest}, where LineNumber is a nonnegative integer and SimpleTest is another simple test object. LineNumber

should indicate the source line of the test. Pairs like this are usually only created via ?_test(...) macros; see Basic macros.

So you could specify your test as for example:

generator_test_() ->
{setup, fun setup/0, fun cleanup/1,
    {inorder,
        [
            fun() -> some_test1() end
            %% OR
            fun some_test1/0
        ]
    }
}.

Besides:

(Note that these macros also have corresponding forms which start with an "_" (underscore) character, as in ?_assert(BoolExpr), that create a "test object" instead of performing the test immediately. This is equivalent to writing ?_test(assert(BoolExpr)), etc.)

And this is why your ?_* test generators work.

EDIT

Examples 2 and 3 are good ways of doing things. Example 2 puts a simple test fun into tests list. Example 3 generates list of 2 tests because

[?_assertEqual(a, b), ?_assertEqual(a == a)]

is actually expanded to

[fun() -> ?assertEqual(a, b) end, fun() -> ?assertEqual(a, a) end]

Use example 2 if you want variables, and example 3 otherwise. Example 1 doesn't give expected results, because "fun() -> some_test1() end" IS a test itself, and it should perform real assertions (?assert*) instead of producing another tests (?_assert*). I hope it's clear now.