13

Consider the following Makefile

CP = .:${HADOOP_HOME}/share/hadoop/common/lib/hadoop-auth-2.2.0.jar:\
${HADOOP_HOME}/share/hadoop/hdfs/hadoop-hdfs-2.2.0.jar:\
${HADOOP_HOME}/share/hadoop/common/hadoop-common-2.2.0.jar:\
${HADOOP_HOME}/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.2.0.jar:\
${HADOOP_HOME}/share/hadoop/mapreduce/lib/hadoop-annotations-2.2.0.jar\

all:
    echo $(CP)

The output of running make is

.:/home/hduser/Hadoop/hadoop-2.2.0/share/hadoop/common/lib/hadoop-auth-2.2.0.jar: /home/hduser/Hadoop/hadoop-2.2.0/share/hadoop/hdfs/hadoop-hdfs-2.2.0.jar: /home/hduser/Hadoop/hadoop-2.2.0/share/hadoop/common/hadoop-common-2.2.0.jar: /home/hduser/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.2.0.jar: /home/hduser/Hadoop/hadoop-2.2.0/share/hadoop/mapreduce/lib/hadoop-annotations-2.2.0.jar

Observe that there are spaces after each :.

Is there a way to define the variable CP with the line breaks, but without the extraneous space substituting every newline?

merlin2011
  • 71,677
  • 44
  • 195
  • 329
  • Does this answer your question? [How to break a string across lines in a makefile without spaces?](https://stackoverflow.com/questions/21246165/how-to-break-a-string-across-lines-in-a-makefile-without-spaces) – imz -- Ivan Zakharyaschev Jun 11 '21 at 23:03

5 Answers5

6

It’s impossible to prevent backslash-newline from becoming a space, and it’s clumsy and error-prone to try to remove the spaces afterwards (what if there are supposed to be spaces?), but you can remove each as it’s produced. This has the significant advantage of working anywhere, even inside function calls. The trick is to embed the space produced in an expression that expands to nothing.

$(call foo) with empty/undefined foo would work, but we can do better: variable names can contain spaces in (GNU) Make. It’s hard to assign to them, but we don’t want to anyway. So then we can shorten it to $(a b) or even $(a ); a backslash-newline will be turned into a space before the lookup. But even a single space works:

foo=bar$(\
)baz

Finally, the parentheses may be omitted for a single-character variable name:

foo=bar$\
baz

…which finally looks like we are (fully) escaping the newline rather than using it somehow. So long as no one assigns to “ ” (which is even crazier than using it!), anyway.

Davis Herring
  • 36,443
  • 4
  • 48
  • 76
  • Thank you for this answer! Just a clarification: Is the concern that someone assigns to a space character or a newline character? – merlin2011 Jun 14 '18 at 03:52
  • It's possible to assign to either one, but here it's the space that matters. It's easy to _read_ `$( )`, but difficult to even read `$(\n)`: there's no escape sequence to give a newline _in_ the variable name. – Davis Herring Jun 14 '18 at 03:56
  • See comments from the GNU Make maintainer in a related question https://stackoverflow.com/a/50863019/579141; though this technique works in current versions of Make, it is not (yet) guaranteed in future versions. A ticket requesting official support for this technique has been submitted. – Michael Henry Jun 17 '18 at 20:49
  • It's a fair concern. Still, while POSIX doesn't require support for spaces in variable names (at least while _creating_ them), its [description of Makefile processing](http://pubs.opengroup.org/onlinepubs/9699919799/utilities/make.html#tag_20_76_13_05) seems pretty clear that there is no other legitimate interpretation here, so change seems unlikely. – Davis Herring Jun 17 '18 at 23:16
  • GNU Make 4.3 now documents the "dollar sign/backslash/newline" method for splitting lines: https://stackoverflow.com/a/50862827/579141 – Michael Henry Feb 23 '20 at 15:25
4

The simplest solution is to use $\<newline> to split the line. Using the following fake paths for brevity:

CP = one$\
     two$\
     three$\
     four

all:
    echo $(CP)

The output will be "onetwothreefour" with no spaces. This is because GNU Make will replace backslash-newline-whitespace with a single space, making the assignment to CP be equivalent to:

CP = one$ two$ three$ four

From https://www.gnu.org/software/make/manual/html_node/Reference.html#Reference: "A dollar sign followed by a character other than a dollar sign, open-parenthesis or open-brace treats that single character as the variable name." So the $<space> pairs are expansions of the variable whose name is a single space character. Since this variable is not defined by default, it will expand to the empty string.

Note that the variable CP will still contain the $<space> pairs until it is expanded. Most of the time, this doesn't matter, but if your makefile depends on using $(value CP) to process the underlying (unexpanded) value, the above technique may provide surprising results.

Also, the recently released GNU Make 4.3 now explicitly documents this technique for splitting lines (https://www.gnu.org/software/make/manual/make.html#Splitting-Lines):

Splitting Without Adding Whitespace

If you need to split a line but do not want any whitespace added, you can utilize a subtle trick: replace your backslash/newline pairs with the three characters dollar sign/backslash/newline:

var := one$\
       word

After make removes the backslash/newline and condenses the following line into a single space, this is equivalent to:

var := one$ word

Then make will perform variable expansion. The variable reference ‘$ ’ refers to a variable with the one-character name “ ” (space) which does not exist, and so expands to the empty string, giving a final assignment which is the equivalent of:

var := oneword

Another technique is to launder the value of CP. This requires GNU Make 3.80 or above because it relies on $(value) and $(eval). It makes use of a few variables and functions for the laundering process.

First, define the variable empty to be the empty string and the variables space and newline to contain a literal space and newline, respectively:

empty :=

space := $(empty) $(empty)

define newline


endef

Next, use $(eval) to programmatically create a recursively expanded variable with a given value:

# Define recursively expanded variable $1 with value $2.
defineVar = $(eval define $1$(newline)$2$(newline)endef)

And define the function resubst to repeated substitute one string with another:

# Replace $1 with $2 in $3 until no more changes are made.
resubst = $\
  $(if $(findstring $1,$3),$\
       $(call resubst,$\
              $1,$\
              $2,$\
              $(subst $1,$2,$3)),$\
       $3)

These are sufficient to define functions two-dimensionally with arbitrary newlines and indentation. The general method comprises three steps:

  1. Prepend a newline to the function body;
  2. Repeatedly replace newline-space pairs with newlines;
  3. Remove all newlines.

The function def removes the newline/whitespace line continuations from the $(value) of a recursively expanded variable defined via define/endef:

# $1 - name of function to redefine as a normalized single-line function.
def = $\
  $(call defineVar,$\
         $1,$\
         $(subst $(newline),$\
                 $(empty),$\
                 $(call resubst,$\
                        $(newline)$(space),$\
                        $(newline),$\
                        $(newline)$(value $1))))

Now def may be used to post-process a recursively expanded variable. For example:

define CP
  one
  two
  three
  four
endef
$(call def,CP)

Now, $(value CP) will return the desired onetwothreefour.

The above is a distillation from my article "GNU Make line continuations": http://drmikehenry.com/gnu-make-line-continuations/

Michael Henry
  • 496
  • 3
  • 8
  • I had this question open in a browser tab while I wrote my blog article last week, and only noticed composing my answer today that @davis-herring had sneaked in a similar answer yesterday. – Michael Henry Jun 14 '18 at 17:25
  • See comments from the GNU Make maintainer in a related question https://stackoverflow.com/a/50863019/579141; though this technique works in current versions of Make, it is not (yet) guaranteed in future versions. A ticket requesting official support for this technique has been submitted. – Michael Henry Jun 17 '18 at 20:49
3

Not really; the backslash-newline combination is defined to produce a space. However, if you are using GNU Make, you could simply $(subst : ,:,$(CP)).

tripleee
  • 175,061
  • 34
  • 275
  • 318
  • I'm on `GNU Make 3.81` but the code you presented does not have the desired effect. The spaces are still there. – merlin2011 Jan 19 '14 at 23:14
  • In particular `echo $(subst ,,$(CP))` produces the same output as `echo $(CP)`. – merlin2011 Jan 19 '14 at 23:16
  • Sorry; added a colon -- try now? – tripleee Jan 19 '14 at 23:16
  • Nope, still same output. My earlier Google queries indicates that spaces are generally tricky in Makefiles, which is why I asked whether there's a way to define it without spaces. – merlin2011 Jan 19 '14 at 23:17
  • There was supposed to be two spaces after `subst` but I can't test from here and am uncertain whether it would work. – tripleee Jan 19 '14 at 23:18
  • Alternatively, put it in `define`...`endef` but that will give you a value with actual newlines, which I understand is not what you want. (I could be wrong; I was wondering about that part of your question.) – tripleee Jan 19 '14 at 23:22
  • 1
    No, I don't want actual newlines. I just want my Makefile to be readable. – merlin2011 Jan 19 '14 at 23:25
2

Sorry to necro this a little bit, but I think this will give you what you're looking for:

CP = .:${HADOOP_HOME}/share/hadoop/common/lib/hadoop-auth-2.2.0.jar:
CP:=$(CP)${HADOOP_HOME}/share/hadoop/hdfs/hadoop-hdfs-2.2.0.jar:
CP:=$(CP)${HADOOP_HOME}/share/hadoop/common/hadoop-common-2.2.0.jar:
CP:=$(CP)${HADOOP_HOME}/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.2.0.jar:
CP:=$(CP)${HADOOP_HOME}/share/hadoop/mapreduce/lib/hadoop-annotations-2.2.0.jar

This should append the next line onto the value of CP, somewhat different than "+=", which adds a space.

goodguy5
  • 422
  • 3
  • 16
  • I the `get *** Recursive variable references itself (eventually). Stop.` error with this too. – wander95 Sep 01 '16 at 21:15
  • @wander95 Interesting.... how many times are you iterating? I've used this method with up to 7 rows, each row being at least 100 characters. I'll try to test out later to see if there's an upper limit on it. – goodguy5 Sep 06 '16 at 18:06
0

I made it similar to @goodguy5

CP:=${HADOOP_HOME}/share/hadoop/common/lib/hadoop-auth-2.2.0.jar
CP+=$(CP)${HADOOP_HOME}/share/hadoop/hdfs/hadoop-hdfs-2.2.0.jar
CP+=$(CP)${HADOOP_HOME}/share/hadoop/common/hadoop-common-2.2.0.jar
CP+=$(CP)${HADOOP_HOME}/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.2.0.jar
CP+=$(CP)${HADOOP_HOME}/share/hadoop/mapreduce/lib/hadoop-annotations-2.2.0.jar
merlin2011
  • 71,677
  • 44
  • 195
  • 329
ItayB
  • 10,377
  • 9
  • 50
  • 77