In Elixir ExUnit, how do I guarantee that the Supervisor will create a new GeNserver?

Question

I'm learning to test in Elixir and this problem appeared:

When I run the following test, sometimes it passes and sometimes don't, I'm thinking it is the fact that the Supervisor don't have the time to restart the GenServer:

  test "Supervisor will revive the dead gensever" do
    {:ok, pid} = KV.Supervisor.start_link([])
    KV.RegistryClient.create KV.RegistryClient, "wallet"
    [h | _] = Supervisor.which_children(pid)
    {KV.RegistryClient, pid_reg, _, _} = h
    send(pid_reg, :insta_kill)

    assert %{active: 1} = Supervisor.count_children(pid)
  end

When occurs, this is the error:

1) test Supervisor will revive the dead gensever (KV.RegistryTest)
     test/kv/registry_test.exs:35
     match (=) failed
     code:  assert %{active: 1} = Supervisor.count_children(pid)
     right: %{active: 0, specs: 1, supervisors: 0, workers: 1}
     stacktrace:
       test/kv/registry_test.exs:41: (test)

How do I prevent this to happen? A timeout is a good approach?

Probably your best bet would be to add a short sleep to your test between the kill and the assertion. — Justin Wood, May 22 '19 at 19:03
Why do you ever want to test _OTP_? Don’t you trust the 40yo battle-proven solution? You should test _your_ code, not the _OTP_ code. The latter works, take it for granted. `ExUnit` does not provide a handy way to test it exactly because nobody is supposed to test _OTP_ internals. — Aleksei Matiushkin, May 23 '19 at 04:49
@AlekseiMatiushkin I'm just trying to study the way OTP works using tests, TDD like — Mateus Luiz, May 24 '19 at 13:09

score 4 · Accepted Answer · answered May 23 '19 at 08:41

The only way you can effectively test this behaviour without race conditions is by:

Making sure the old process is dead. This can be done by monitoring the process before you send the kill signal and then assert_receive {:DOWN, ^monitor_ref, _, _, _}
Query the supervisor until the active count changes to one. This can be done by executing a function once every 10ms or so

However, as others have said, this behaviour is guaranteed by Erlang/OTP. Therefore, instead of testing if a supervisor is actually restarting something, I would rather test if you are passing the proper child specification to the supervisor. So assuming the supervisor starts a processed based on KV.Registered, I would do this:

assert %{restart: :permanent} = Supervisor.child_spec(KV.Registered)

In other words, I would test the contract with supervisors, without testing the supervisors themselves.

Thanks for the feedback, I'll try this – Mateus Luiz May 24 '19 at 13:12 — Mateus Luiz, May 24 '19 at 13:12

score 2 · Answer 2 · answered May 23 '19 at 06:11

It is a timing issue. The short version is don't bother testing OTP. IT is very well tested already.

But if you have a case in the future where you need to ensure startups work correctly understanding how Supervisors start servers is useful, see this video https://www.youtube.com/watch?v=2i_XrY5CCXE

In Elixir ExUnit, how do I guarantee that the Supervisor will create a new GeNserver?

2 Answers2