Check all test files are compilable

Question

I have a case when I need to check that all files in my app's test suite are compilable.

I'm trying to create a list of all files and then compile them, but it turns out I can't do it without starting ExUnit.

find apps/*/test -name "*.exs" | xargs elixirc

== Compilation error in file apps/api/test/api/request_validators/validator_test.exs ==
** (RuntimeError) cannot use ExUnit.Case without starting the ExUnit application, please call ExUnit.start() or explicitly start the :ex_unit app
    expanding macro: ExUnit.Case.__using__/1

How to check that all test files are compilable without actually running tests?

score 2 · Accepted Answer · answered Dec 24 '19 at 13:36

2

The proper solution would be to indeed start ExUnit, compile all the files and examine the outcome.

mix run -e 'ExUnit.start(); exit({:shutdown, (if match?({:ok, _, _}, Kernel.ParallelCompiler.compile(Path.wildcard("test/**/*.exs"))), do: 0, else: 1)})'

That way you might also explicitly specify the error code to return.

answered Dec 24 '19 at 13:36

Aleksei Matiushkin

119,336
10
100
160

It works faster than my approach, thanks! I had to change the path to `"apps/*/test/**/*.exs"` for it to work, because I'm running tests in an umbrella app. – denis.peplin Dec 24 '19 at 14:12
It's stopped working for me. However, the error is too specific to share it here. – denis.peplin Dec 26 '19 at 15:19
This code is indeed bullet-proof. I am positive the issue is induced. – Aleksei Matiushkin Dec 26 '19 at 15:20
The error is about missing configuration in one of the umbrella apps when compiling another. There are no such errors when I just run tests. – denis.peplin Dec 26 '19 at 15:24
Ah, umbrella, yes. You probably should check how [tag:elixir] compiler deals with umbrellas and if there are some specific quirks there. – Aleksei Matiushkin Dec 26 '19 at 15:26

denis.peplin · Answer 2 · 2019-12-24T15:35:23.790

1

I've found an answer.

Just run tests with some tag that doesn't exist. This will compile all tests but none of them will be actually run

mix test --only whatever

UPDATE: this only works for umbrella apps. For a regular app, nonexistent tag leads to an error code 1 (see comments to this question).

It's possible to create a workaround: have an empty test that is always green and mark it with the tag.

See also the accepted answer for a better solution.

edited Dec 24 '19 at 15:35

answered Dec 24 '19 at 13:24

denis.peplin

9,585
3
48
55

This works if you visually inspect the output, but the status of the unix status of the command is 1, so might fail if you use it in a script. See my answer for an alternate approach that tries to circumvent that. – Paweł Obrok Dec 24 '19 at 13:27
Yes, the status is 1, and that means failure. 0 status is for success. I'm not inspecting it visually, CircleCI correctly emits red status when I'm doing that. – denis.peplin Dec 24 '19 at 14:08
@denis.peplin in this case, the status would be `1` no matter whether the compilation succeeded or not because inexisting tag is already considered as a failure. That said, the above simply does not check anything. – Aleksei Matiushkin Dec 24 '19 at 15:03
I've just checked that it in two cases: a regular app and an umbrella app. It emits `1` when it's a regular app and `0` when it's an umbrella. – denis.peplin Dec 24 '19 at 15:20
Interesting. I would consider this being a bug worth reporting. – Aleksei Matiushkin Dec 24 '19 at 15:27
Maybe it is, reported if you feel like it. There is a workaround for the problem: I can create one empty test that is always green and mark it with the tag. This way, my answer is still a working solution. But I will use your anyway because it works faster. – denis.peplin Dec 24 '19 at 15:33

Check all test files are compilable

2 Answers2