Using the call function correctly in a makefile

Question

I am trying to compile for different software directories with different optimization levels etc. I created the following makefile to do so:

OWNER           = betsy molly fred
DOG             = poodle mutt doberman
COLOUR          = brown red yellow
ATTR            = big small
LEGS            = 0 3

#we want every possible combination to be excercised
OUTPUT_STUFF = $(foreach own,$(OWNER),$(foreach dog,$(DOG),$(foreach col,$(COLOUR),$(foreach attr,$(ATTR),$(foreach legs,$(LEGS),new/$(own)/$(dog)/$(col)/$(attr)/$(legs)/dogInfo.txt)))))

.PHONY: all

all: $(OUTPUT_STUFF)

define PROGRAM_template

own             = $(1)
dog             = $(2)
col             = $(3)
attr            = $(4)
legs            = $(5)

BUILD_DIR           = new/$(own)/$(dog)/$(col)/$(attr)/$(legs)

#for each build directory, we are going to put a file in it containing the build dir. string
$$(BUILD_DIR)/dogInfo.txt:
    @echo "$$@"
    mkdir $$(BUILD_DIR)
    @echo "$$(BUILD_DIR)" > $$(BUILD_DIR)/dogInfo.txt
endef

#call the function many times
$(foreach own,$(OWNER),$(foreach dog,$(DOG),$(foreach col,$(COLOUR),$(foreach attr,$(ATTR),$(foreach legs,$(LEGS),$(eval $(call PROGRAM_template,$(own),$(dog),$(col),$(attr),$(legs))))))))

As you can see, this simple test program loops through different combinations of owner, dog etc. The end goal is to have a directory, new, that has all owners as dirs, and in those, all dogs, etc. At the bottom is just a file with the path in it.

When I run this, the output is:

new/betsy/poodle/brown/big/0/dogInfo.txt
mkdir new/fred/doberman/yellow/small/3
mkdir: cannot create directory `new/fred/doberman/yellow/small/3': No such file or directory
make: *** [new/betsy/poodle/brown/big/0/dogInfo.txt] Error 1

So, for some reason, the target is ok, but the seemingly exact same variable is the last in my loops. Fundamentally, I don't understand what is happening that well.

Weird foreach + user-defined function behavior in Makefiles seems to answer, but I don't fully get it. In my mind, when the function is called, it fills in all instances with one $, and the escaped ones become $(BUILD_DIR). It then 'pastes' the code to the temporary file, and after it's done all the calls it evaluates the file, substituting the variables as normal.

One (ugly) solution I thought of is to make the BUILD_DIR variable different every time like so:

B_D_$(1)_$(2)_$(3)_$(4)_$(5) = ~~~

Answer 1

Alex is correct (although I think he means recipe, not receipt :-)). The best way to debug complex eval issues is to replace the eval function with a call to info instead. So if you have something like:

$(foreach A,$(STUFF),$(eval $(call func,$A)))

then you can rewrite this as:

$(foreach A,$(STUFF),$(info $(call func,$A)))

Now make will print out to you exactly what the eval is going to parse. It's usually pretty clear, looking at the makefile output, what the problem is. In your case you'll see something like this in the output (leaving out all the extra variable settings):

BUILD_DIR = new/betsy/poodle/brown/big/0
$(BUILD_DIR)/dogInfo.txt:
            @echo "$$@"
            mkdir $(BUILD_DIR)
            @echo "$(BUILD_DIR)" > $(BUILD_DIR)/dogInfo.txt

BUILD_DIR = new/betsy/poodle/brown/big/3
$(BUILD_DIR)/dogInfo.txt:
            @echo "$$@"
            mkdir $(BUILD_DIR)
            @echo "$(BUILD_DIR)" > $(BUILD_DIR)/dogInfo.txt

etc. Notice how you're setting the global variable BUILD_DIR every time. In make, variables have only one value (at a time). While make is reading the makefile it expands the target and prerequisite lists immediately, so whatever value BUILD_DIR has at that time will be used for targets/prerequisites, so this works for you.

But when make finishes reading the makefile, the value of BUILD_DIR will always be the last thing you set it to; in this case new/fred/doberman/yellow/small/3. Now make starts to invoke the recipes for each target, and when it does that it will expand BUILD_DIR in the recipes then, and so ALL the recipes will get that same value.

As Alex points out, you should ensure that your recipe uses only automatic variables like $@, which are set correctly for each rule. If you do that you'll notice that you don't really need to redefine the rule at all because it's actually the same recipe for all the targets. And if you notice THAT, you'll notice you don't need the whole eval or call complexity in the first place.

All you have to do is compute the names of all the targets, then write a single rule:

ALLDOGINFO = $(foreach own,$(OWNER),$(foreach dog,$(DOG),$(foreach col,$(COLOUR),$(foreach attr,$(ATTR),$(foreach legs,$(LEGS),new/$(own)/$(dog)/$(col)/$(attr)/$(legs)/dogInfo.txt)))))

$(ALLDOGINFO):
        @echo "$@"
        mkdir $(dir $@)
        @echo "$(dir $@)" > $@

If you don't want the trailing slash you have to use $(patsubst %/,%,$(dir $@)) instead.

Answer 2

The problem is that when $$(BUILD_DIR) is evaluated in receipt, the loop is already complete. The solution is to rewrite the receipt:

$$(BUILD_DIR)/dogInfo.txt:
    @echo "$$@"
    mkdir $$(@D)
    @echo "$$(@D)" > $$@

Answer 3

I don't think your problem is necessarily with something to do with make. This command:

mkdir new/fred/doberman/yellow/small/3

will fail if one of the parent directories (for example, yellow) doesn't already exist. The error it spits out in this case is the one you're getting, so it seems likely this is the case. If you want a command that makes all parent directories of a given directory as needed, you should run mkdir -p, like this:

mkdir -p $$(BUILD_DIR)

See the mkdir man page for a full description of what -p does.