Dependency ordering error for multi-job make

Question

The Makefile below has to create (multiple) output directories, and generate output in those directories, from input in the directory above. So, on entry, dirn exists, and dirn/file.foo exists. The build has to create dirn/out/file.bar.

This Makefile works when run as a single job (note that it creates the two required source directories and files in the $(shell)). It presumably works because makedirs is the first/leftmost prerequisite for all. However, it doesn't work for a multi-job make (ie. make -j4/whatever).

Any ideas on how to fix the dependencies to ensure that the output directories are made before they're required?

EDIT

I should have made it clear that I have tried various order-only prerequisite solutions, but I couldn't do this and guarantee that the target actually rebuilt (the point of order-only is generally to prevent rebuilding, not enforce dependency ordering). If you have an OO solution, please check it! Thanks.

# expected output:
# made directories
# copying dir1/out/../file.foo to dir1/out/file.bar
# copying dir2/out/../file.foo to dir2/out/file.bar
# created all output files
# done

    $(shell mkdir dir1 >& /dev/null; touch dir1/file.foo; \
            mkdir dir2 >& /dev/null; touch dir2/file.foo)

    OUTDIRS = dir1/out dir2/out
    OUTPUTS = dir1/out/file.bar dir2/out/file.bar 

    .DEFAULT_GOAL := all

    .PHONY: makedirs $(OUTDIRS)

    .SUFFIXES: .foo .bar

    %.bar : ../%.foo
        @echo "copying $< to $@"
        @cp $< $@

    all : makedirs outputs
        @echo "done"

    outputs : $(OUTPUTS)
        @echo "created all output files"

    makedirs : $(OUTDIRS)
        @mkdir -p $(OUTDIRS)
        @echo "made directories"

    clean :
        @rm -rf dir1 dir2

Answer 1

make $(OUTPUTS) have an order-only dependency on the directories themselves:

 $(OUTDIRS) : 
      mkdir -p $@

 $(OUTPUTS) : | $(OUTDIRS)

This will guarentee that the directories are created before $(OUTPUTS), but will not cause the outputs to rebuild if the directories are newer than the targets (which is important, as a directories' time stamp is set each time a file is added to it...).

Note: you can also add a mkdir -p in your output recipe's which will create the directory if it's not already there each time you run an output rule, but I prefer the above method.

Note2: in your existing makefile you could also just add a line: $(OUTPUTS): makedirs, which would force the makedirs rule to be run before any of the outputs are built, but again, I prefer the above solution :-)

---- EDIT: -----

Something is odd then -- what version of make are you using? I just ran the following: notice the sleep 1's when making the directories, which means if there was a concurrency issue, it would definitely have hit:

$(shell mkdir dir1 >& /dev/null; touch dir1/file.foo; \
        mkdir dir2 >& /dev/null; touch dir2/file.foo)


OUTDIRS = dir1/out dir2/out
OUTPUTS = dir1/out/file.bar dir2/out/file.bar

.DEFAULT_GOAL := all

$(OUTPUTS) : | $(OUTDIRS)

$(OUTDIRS) :
        @echo "making $@"
        sleep 1
        mkdir -p $@
        @echo "done making $@"


%.bar : ../%.foo
        @echo "copying $< to $@"
        @cp $< $@

all : outputs
        @echo "done $@"

outputs : $(OUTPUTS)
        @echo "created all output files"

clean :
        @rm -rf dir1 dir2

And the output was:

~/sandbox/tmp20> make -j -f Makefile2
making dir1/out
making dir2/out
sleep 1
sleep 1
mkdir -p dir1/out
mkdir -p dir2/out
done making dir1/out
done making dir2/out
copying dir1/out/../file.foo to dir1/out/file.bar
copying dir2/out/../file.foo to dir2/out/file.bar
created all output files
done all

Notice it concurrently builds dir1/out and dir2/out, and doesn't run the pattern rules until both are finished. I also verified that the solution I mentioned in note2 also works (at least on my machine...).

When doing pattern rules, you can specify dependencies outside of the pattern, so you can do:

foo.o: foo.h

%.o: %.c
    recipe here...

which will rebuild foo.o if foo.h is newer, or try to build foo.h if it doesn't exist before foo.o is built.