1

I'm having trouble understanding how Makefile deals with variable expansion when a rule creates a file that should modify the behaviour of a wildcard expansion.

For instance, I have the following Makefile:

TEXT_FILES = $(wildcard *.txt)

all: create test

create:
    @touch hello.txt

test:
    @printf "The files are $(TEXT_FILES)\n"

What I expected to get when running make is

The files are hello.txt

because TEXT_FILES should be evaluated when it is called, not when defined. However, I actually get (running make twice)

The files are (variable is empty when executing for the first time)

The files are hello.txt (found next time)

How can I get the desired behavior?

Community
  • 1
  • 1
mbrandalero
  • 386
  • 1
  • 3
  • 15
  • Found this possibly related issue here http://stackoverflow.com/questions/22630996/makefile-sequential-execution Still, no solution. – mbrandalero Jan 04 '17 at 19:24
  • What is your *exact* use-case? Maybe it's a XY-problem. – uzsolt Jan 04 '17 at 19:31
  • I have a folder with templates in C code that are transformed 1 - 1 to something else (also in C), but additional files may be generated in the process Found this: http://stackoverflow.com/questions/9744865/how-to-force-revaluation-of-variables-in-makefile Apparently wildcard expansion takes place long before rules are evaluated, so there's no way to achieve the behavior I want – mbrandalero Jan 04 '17 at 19:37
  • And what do you want to do with "additional files"? Do you know anything about their names: can you "extract" from your templates? Or the names of these generated files are randomly? – uzsolt Jan 05 '17 at 09:36

1 Answers1

3

This behavior is the result of make's internal cache. For performance reasons, make will cache the state of the filesystem that it believes to be true, based on what it was before plus the changes it knows have been made (because of rules that have been invoked).

Functions like wildcard will query the internal cache, if it's available.

However, in your situation you're not describing the actual behavior of your rules to make: you didn't tell make that this rule generates the extra file. As a result make won't update its cache and the wildcard function never finds this secret file.

You don't say what you want to do precisely, but why don't you just use the shell's wildcarding facilities instead of make's wildcard function? The shell obviously has no idea about make's internal caches:

TEXT_FILES = *.txt

all: create test

create:
        @touch hello.txt

test:
        @echo The files are $(TEXT_FILES)
MadScientist
  • 92,819
  • 9
  • 109
  • 136
  • I should point out that this makefile is not parallelism-safe: you can't use make's `-j` option and have it work reliably. You'd need to make `create` a prerequisite of `test` for that to work. – MadScientist Jan 05 '17 at 16:46
  • That worked! Thanks. So it seems that while make's `wildcard` function is evaluated when make is parsed, the shell's wildcard`*` is expanded only afterwards, when the `TEXT_FILES` variable is expanded. Is that correct? – mbrandalero Jan 05 '17 at 17:00
  • 1
    Not really. Make's functions, including wildcard, are expanded depending on where they appear in the makefile, see https://www.gnu.org/software/make/manual/html_node/Reading-Makefiles.html for the rules. Since `wildcard` in your example appears in the recipe, it's not expanded until make is just about to run that particular recipe (not when the makefile is parsed). However, because of make's caching behavior as I described above, the wildcard function only matches files that existed when make started or make knows was created. This is specific to the wildcard function. – MadScientist Jan 05 '17 at 18:08
  • Witihout using the wildcard function make won't try to expand `*` at all; it's just another character to make, like `a` or whatever. Make only cares about things that might be variables (preceded by `$`). So make passes the string `echo The files are *.txt` to the shell, and the shell is responsible for handling the file globbing. – MadScientist Jan 05 '17 at 18:10
  • Just to note that starting with GNU make 4.4 this issue is no longer present. Whenever GNU make invokes a command that might modify the filesystem it dirties its internal cache so it will re-visit the contents of any directories. This probably makes things slower, but is simpler to reason about. – MadScientist Dec 20 '22 at 15:23