How can I clean up after an error in a Makefile?

Question

foobar may create the output file even when it fails, so I need to delete it in that case.

I can do this:

foo: bar baz
        foobar $^ -o $@ || (rm -f $@ && exit 1)

But this does not propagate the same exit code returned by foobar (which is then outputted by make). Is there any way to catch the error in the Makefile rather than in the shell?

Christian Hujer · Answer 1 · 2018-12-03T09:46:38.070

In case DELETE_ON_ERROR doesn't cut it, and what you're looking is a bit like a tearDown, @After, or finally in Java/JUnit, this is what you can do:

Use .ONESHELL: to have all shell commands executed in a single shell.
Install a trap for EXIT to get your cleanup done.
Make sure that the shell exits in case of any errors by setting errexit. While we're at it, we can also set pipefail right away.

For example, let's say you want to start a docker container, do a test, stop the docker contaner no matter what, but get the result of the test. This is how to do it:

export SHELL:=/bin/bash
export SHELLOPTS:=$(if $(SHELLOPTS),$(SHELLOPTS):)pipefail:errexit

.ONESHELL:

.PHONY: test
test:
    function tearDown {
        docker stop test-image
    }
    trap tearDown EXIT
    docker run --name test-image …
    testStep1…
    testStep2…
    testStep3…
    …

How it works

The export SHELL export tells GNU make to use bash as shell, which has heavier footprint than the default sh but way more features.
The export SHELLOPTS sets the pipefail and errexit flags for the bash shell.
- pipefail ensures that the exit status of a pipe will not be the last command but the last with a non-zero exit status. So, false | true would return 1 instead of 0.
- errexit ensures that the exit status of a command sequence will not be the last command but the last with a non-zero exit status, and that subsequent commands will not be executed. So, false ; true would return 1 instead of 0 and true would not be executed.
The .ONESHELL: tells GNU make to run all the commands in a single shell. That means, your recipe really is one shellscript now. (Requires GNU make 3.82 or later.)
The function tearDown { docker stop test-image } defines a shell function named tearDown. In this example, it will stop the docker container.
The trap tearDown EXIT is the most crucial part of everything in this example. It tells the shell that was invoked for the recipe to run the tearDown function on exit, that is, no matter whether the commands were successful or failed.

Limitations

This is analogous to finally in Java. Reuse across multiple targets / tests is not possible. It is definitely not like @AfterClass / @AfterAll or tearDown() / @After / @AfterEach in JUnit.

Getting it even better

But you could do that, in case you need it. Say, you want to run multiple tests on the same docker container, and tear it down no matter what. That would be analogous to @AfterClass / @AfterAll in JUnit. Then that could look like this:

export SHELL:=/bin/bash
export SHELLOPTS:=$(if $(SHELLOPTS),$(SHELLOPTS):)pipefail:errexit

.ONESHELL:

.PHONY: start
start:
    docker run --name test-image …

.PHONY: stop
stop:
    docker stop test-image

.PHONY: test
test: start
    function tearDown {
        $(MAKE) stop
    }
    trap tearDown EXIT
    $(MAKE) -k testImpl

.PHONY: testImpl
testImpl: testCase1 testCase2 testCase3

.PHONY: testCase1
testCase1:
    …

.PHONY: testCase2
testCase2:
    …

.PHONY: testCase3
testCase3:
    …

This would now run all tests, even if the first one failed, clean up after all of them have finished, and report an error if any of the tests failed.

Disclaimer: This requires the .ONESHELL feature of GNU make, which was introduced in GNU make 3.82. The current version of GNU make as of this edit is GNU make 4.2.1, and Mac OS X still ships with GNU make 3.81.

A word of warning: .ONESHELL may make your recipes behave differently. See the last paragraph of [the documentation](https://www.gnu.org/software/make/manual/html_node/One-Shell.html). — schieferstapel, May 22 '19 at 09:13
@schieferstapel Indeed, that's why the line about `SHELLOPTS` is crucial. There's a reason why `.ONESHELL` isn't default, and that reason is that `errexit` is not default for shells and there's no standard way to set it, it depends on the shell. — Christian Hujer, May 23 '19 at 05:45
Thanks for elaborating. But even with `set -e`, recipes may behave differently, e.g. `false ; true`, where `true` won't get executed any more. Nothing against your solution at all, but one should be aware of the implications of `.ONESHELL`. — schieferstapel, May 24 '19 at 06:40

score 9 · Answer 2 · edited Oct 27 '18 at 11:38

9

Does .DELETE_ON_ERROR: do what you want here?

From Errors in Recipes:

Usually when a recipe line fails, if it has changed the target file at all, the file is corrupted and cannot be used—or at least it is not completely updated. Yet the file’s time stamp says that it is now up to date, so the next time make runs, it will not try to update that file. The situation is just the same as when the shell is killed by a signal; see Interrupts. So generally the right thing to do is to delete the target file if the recipe fails after beginning to change the file. make will do this if .DELETE_ON_ERROR appears as a target. This is almost always what you want make to do, but it is not historical practice; so for compatibility, you must explicitly request it.

If not, or if you only need it for that one target, then the shell line you want is:

foobar $^ -o $@ || (ret=$$?; rm -f $@ && exit $$ret)

edited Oct 27 '18 at 11:38

Jakub Kukul

12,032
3
54
53

answered Feb 19 '15 at 03:14

Etan Reisner

77,877
8
106
148

I guess it wouldn't hurt in this case to use it for all targets. But for future reference, is there a way to do it without polluting the shell command? It's nice to see simple clean lines echoed by `make` rather than long shell hacks. Unless there is a way to suppress echoing on *part* of a shell command, which I doubt. – Matt Feb 19 '15 at 03:36
1

That option is all-or-nothing. I don't know of any way to do it for just specific rules. This sort of shell game is the best I've got for that. That being said you can play around with automake-style [silent rules](https://gist.github.com/deryni/017dc953c75b9396b80d) to control what your make output looks like even in the face of "ugly" shell commands. – Etan Reisner Feb 19 '15 at 05:08
@matt You can move the shell code to a script to make it prettier. You can also prefix it with something like `@echo foobar $^ -o $@; ...` but this may confuse people who look at the make output and wonder how the file got deleted. – Ross Ridge Feb 19 '15 at 20:16
@RossRidge That second suggestion is my silent rules suggestion, just for the record. – Etan Reisner Feb 19 '15 at 21:12
My automake-style silent rules functionality now has a github project: https://github.com/deryni/silent-make-rules . – Etan Reisner Jun 29 '15 at 14:06
`.DELETE_ON_ERRROR` should be the default IMHO. You nearly always want it. The only occasion I can think of that you want to keep a broken output file is if you need to see it to debug a broken build. – bobbogo Oct 31 '18 at 10:17
Note that the shell solution also works for removing intermediate directories on error. Instead of repeating the line over and over again, one can define a variable `cleanup = { rc=$$?; rm -rf $1; exit $$rc; }` and then in recipe lines use something like ` || $(call cleanup,)`. – schieferstapel May 22 '19 at 09:22

How can I clean up after an error in a Makefile?

2 Answers2

How it works

Limitations

Getting it even better