Goal
I want to write a test that runs as part of dune test
but does not run
as part of dune build
(for example, because it is slow). This test
should have an expected output file and compare the actual output to it, and dune promote
should be able to update the expected output.
First attempt: implicit rules
Following the instructions in the
manual,
I created the following minimal diff
test:
dune-project
:
(lang dune 3.3)
dune
:
(test (name mytest))
mytest.ml
:
let () =
Printf.printf "this is mytest\n"
mytest.expected
:
xxx
Since mytest.expected
exists, dune
automatically creates a diff
test. However, it runs mytest.exe > mytest.output
as part of
dune build
:
$ dune clean
$ dune build --verbose
[...]
Running[4]: (cd _build/default && ./mytest.exe) > _build/default/mytest.output
$ ls _build/default/
mytest.exe* mytest.ml mytest.mli mytest.output
Then when I run dune test
, it does the comparison to the output file
it already made:
$ dune test --verbose
[...]
Command [1] exited with code 1:
$ /usr/bin/git --no-pager diff --no-index --color=always -u _build/default/mytest.expected _build/default/mytest.output
diff --git a/_build/default/mytest.expected b/_build/default/mytest.output
index d6459e0..9dac0ea 100644
--- a/_build/default/mytest.expected
+++ b/_build/default/mytest.output
@@ -1 +1 @@
-xxx
+this is mytest
My goal is for dune build
to not run the test, and only run it with
dune test
.
Second attempt: explicit rules
I tried explicitly writing my own rules in the dune
file, also closely following what is in the manual:
(test
(name mytest)
)
; ADDED
(rule
(with-stdout-to tests.output
(run ./mytest.exe)))
; ADDED
(rule
(alias runtest)
(action
(diff tests.expected tests.output)
)
)
but the same thing happens: tests.output
is created by dune build
.
(I also created tests.expected
.)
Third attempt: more (alias runtest)
?
I tried making the generation rule part of runtest
:
(test
(name mytest)
)
(rule
(alias runtest) ; ADDED
(action ; ADDED 'action' wrapper
(with-stdout-to tests.output
(run ./mytest.exe)
)
)
)
(rule
(alias runtest)
(action
(diff tests.expected tests.output)
)
)
but this still creates tests.output
during dune build
.
Fourth attempt: enabled_if
I looked at enabled_if
but didn't see any promising condition to test.
Fifth attempt: one rule
I tried putting all of the logic into one (alias runtest)
rule:
(test
(name mytest)
)
; COMBINED
(rule
(alias runtest)
(action
(progn
(with-stdout-to tests.output
(run ./mytest.exe)
)
(diff tests.expected tests.output)
)
)
)
but not only does the test still run during dune build
(why?), but now
dune promote
fails if run afterward:
$ dune promote
Skipping promotion of _build/default/tests.output to tests.expected as the
file is missing.
A clue: tests that have output files always run during the build
After further experimentation, I found that this dune
file:
(rule
(alias runtest)
(action
(bash "echo test with no saved output")
)
)
(rule
(alias runtest)
(action
(with-outputs-to outputs.txt
(bash "echo test that saves output to a file")
)
)
)
will run the first test as part of dune test
but the second as part of dune build
. So my working hypothesis is any rule that has any output file will run with dune build
. That in turn makes it impossible to run a diff
test as part of dune test
(and not dune build
).