Understanding bjam's targets, and how to specify new ones?

Question

I'm having problems understanding how to specify and invoke targets with bjam. By this, I mean that I want to provide bjam with command-line targets to build (from a Makefile actually) that correspond to different aspects of the build process, rather than just running the whole thing.

For example, right now when I type 'bjam' it goes off and builds a python extension, runs a unit-test file, and also creates a separate 'main' executable. I have custom rules that do each step, and my Jamfile simply lists them in order:

project-name = example ;

sources =
  $(project-name).cpp
  $(project-name)_ext.cpp
  ;

build-ext $(project-name) : $(sources) ;

build-main $(project-name) ;

In my Jamroot (up one directory) I have these rules defined, here's the incomplete file:

# A rule to simplify declaration of extension tests:
rule run-test ( test-name : sources + )
{
    import testing ;
    testing.make-test run-pyd : $(sources) : : $(test-name) ;
}

# A rule to further simply declaration of extension tests:
rule run-ext-test ( project-name )
{
  run-test $(project-name) : $(project-name)_ext test_$(project-name)_ext.py ;
}

# A rule to simplify copying of the extension and Boost.Python libraries to the current directory
rule convenient-copy ( project-name )
{
  install convenient_copy
    : $(project-name)_ext
    : <install-dependencies>on <install-type>SHARED_LIB <install-type>PYTHON_EXTENSION
      <location>.
    ;
}

rule build-ext ( project-name : sources + )
{
  python-extension $(project-name)_ext : $(sources) : ;

  # copy the extension and Boost.Python libraries to the current directory
  convenient-copy $(project-name) ;

  # run extension tests
  run-ext-test $(project-name) ;
}

rule build-main ( project-name : other-sources * )
{
  obj $(project-name).o : $(project-name).cpp ;
  exe main_$(project-name) : main_$(project-name).cpp $(project-name).o $(other-sources) ;
  install main : main_$(project-name) : <location>. ;
}

However I've noticed that the following invocations of bjam don't do what I'd like them to do:

$ bjam build-main
notice: could not find main target build-main
notice: assuming it is a name of file to create.
don't know how to make <e>build-main
...found 1 target...
...can't find 1 target...

$ bjam main_example
...patience...
...patience...
...found 1597 targets...
...updating 3 targets...
gcc.compile.c++ bin/gcc-4.6/debug/main_example.o
gcc.compile.c++ bin/gcc-4.6/debug/example.o
gcc.link bin/gcc-4.6/debug/main_example
...updated 3 targets...

^^^ but the install rule wasn't run so the binary isn't copied to the Jamfile directory.

Oddly, there are some targets that do things, but not always what I expect:

$ bjam main
...patience...
...patience...
...found 1598 targets...
...updating 3 targets...
gcc.compile.c++ bin/gcc-4.6/debug/main_example.o
gcc.compile.c++ bin/gcc-4.6/debug/example.o
gcc.link main_example
...updated 3 targets...

That did create the binary, in the Jamfile directory.

Where did the main target come from? I didn't define it...

Another odd one:

$ bjam example_ext
...patience...
...patience...
...found 2834 targets...
...updating 3 targets...
gcc.compile.c++ bin/gcc-4.6/debug/example.o
gcc.compile.c++ bin/gcc-4.6/debug/example_ext.o
gcc.link.dll bin/gcc-4.6/debug/example_ext.so
...updated 3 targets...

^^^ created example_ext.so, but didn't copy it to the Jamfile location.

$ bjam example_ext.so
notice: could not find main target example_ext.so
notice: assuming it is a name of file to create.
...patience...
...patience...
...found 2836 targets...
...updating 4 targets...
gcc.compile.c++ bin/gcc-4.6/debug/example.o
gcc.compile.c++ bin/gcc-4.6/debug/example_ext.o
gcc.link.dll bin/gcc-4.6/debug/example_ext.so
common.copy example_ext.so
...updated 4 targets...

^^^ created the .so file and copied it, but didn't invoke convenient-copy to bring in the libboost_python.so files.

I really don't understand what's going on here. The bjam documentation is really causing me serious problems. It describes targets in detail, but in the context of rules, not in the context of invoking bjam from the command line. I did come across some mention of pseudotargets and 'generate' but it seemed far too complicated for what I feel should be a simple use-case. There was also mention of a 'bind' mechanism but the documentation mentions =$(BINDRULE[1])= which makes no sense to me.

I also came across aliases, NOTFILE and explicit but I wasn't sure if I was on the right track, and wasn't able to do anything conclusive.

Are there any good examples of how to create custom targets in bjam? Or am I simply trying to use bjam in ways that weren't intended?

score 5 · Answer 1 · answered Oct 02 '12 at 09:46

A bjam invocation with no arguments builds everything because you don't have any target marked as explicit, which can be used to prevent some targets building unless explicitly requested.

bjam build-main fails as build-main is not a name of a target or file to generate; it is name of a rule (function) that can be called with different arguments, each call declaring different sets of targets.

bjam main_example makes the target that was declared with:

exe main_$(project-name) : main_$(project-name).cpp $(project-name).o $(other-sources) ;

which naturally doesn't incorporate the install target for it, declared on the next line.

bjam main builds main_example and installs it, because main is the name of its install target, declared with: install main : main_$(project-name) : <location>. ;

Note that had you called build-main more than once in your jamfile, every bjam invocation would exit with error: No best alternative for ./main, so it's better to rename the name of the install target to something like install_main_$(project-name) to prevent name collisions. Then bjam install_main_example will build and install main_example.

bjam example_ext makes the target that was declared with:

python-extension $(project-name)_ext : $(sources) : ;

and no install again like with bjam main_example.

bjam example_ext.so works as example_ext.so is indeed a file name that is being created (under the given platform of course), so all targets that result in files named example_ext.so will be generated by that invocation. Now this is why not all files that convenient-copy instructed to be installed were installed by bjam example_ext.so invocation. Here I want to clarify something: "didn't invoke convenient-copy" isn't quite an accurate terminology. convenient-copy is a name of a rule, not a target, and with the code as written above, that rule will always get called regardless of bjam invocation arguments. All it does when called is declaring a target named convenient_copy (note the underscore), which in turn results in (implicitly) declaring a few file targets (for install) e.g. example_ext.so, libboost_python.so and other dependencies that are matched by <install-type>SHARED_LIB <install-type>PYTHON_EXTENSION. When convenient-copy rule gets called, it doesn't actually build anything, it just declares some targets. What gets actually built is decided at a later phase, and the decision depends on bjam invocation arguments.

bjam convenient_copy would build example_ext.so and install it properly together with its dependencies, yet it suffers from the same problem that main install target does: it will break if you call convenient-copy more than once. Changing the name from convenient_copy to install_$(project-name)_ext would solve the issue, and then you would invoke the install with bjam install_example_ext.

And finally if you want a target to not build on bjam invocation with no arguments, you mark that target as explicit, e.g.

explicit install_main_$(project-name) ;

would suppress main_example's installation unless explicitly requested. To prevent main_example's build too, add explicit for main_$(project-name) and $(project-name).o as well:

explicit main_$(project-name) ;
explicit $(project-name).o ;

or all together:

explicit install_main_$(project-name) main_$(project-name) $(project-name).o ;

Note that declaring main_$(project-name) as explicit while not doing so for install_main_$(project-name), or declaring $(project-name).o as explicit while not doing so for main_$(project-name) doesn't make sense, as if a target is requested to build, all its dependency targets will be requested to build as well, even when those dependencies are explicit.

Thanks for this, this is very helpful. Just to clarify something - rules are not *executed* sequentially, they simply provide a container for a set of target declarations? Is that right? So in my build-ext rule, it doesn't invoke those 3 rules sequentially, but instead adds them to some sort of global dependency tree that is then resolved after processing the Jamfile? Is that right or am I misleading myself? There's something fundamental about Jamfiles that I'm just not getting, maybe it's this... — davidA, Oct 10 '12 at 02:19
Rules are just functions. All rules in your example declare targets, and it's a very common use of rules, but it's not mandatory for a rule to declare target(s). A rule can just print something, do an arithmetic computation, or compute a set of conditional properties - whatever, it's just a function. And as such, a rule gets executed only if it gets called. In your example, build-ext rule gets executed only because you call it in the Jamfile: `build-ext $(project-name) : $(sources) ;`. Yes, that line is a function call, and the colon is used to separate the arguments. — usta, Oct 10 '12 at 05:34
The rules themselves are executed pretty much in the intuitive predictable order. That still doesn't have anything to do with the order *targets* get built. When a target is declared in a rule (or outside of any rule, doesn't make a difference), it doesn't get built immediately, instead it gets added in some global container of targets like you said. And then after the execution of rules is over, it gets decided what targets to build, and in what order. That order is restrained by the dependencies between targets, but otherwise is not predictable or to be relied upon. — usta, Oct 10 '12 at 05:49
So dependencies is how one controls the build order of targets. Aside note: there is a way to force a target to be built immediately, during rule execution phase, but that is more of an advanced topic. There are also facilities that make use of that immediate build thing in their implementation, e.g. `configure.check-target-builds` (using which already isn't a very advanced thing to do :) ) — usta, Oct 10 '12 at 06:09
If a rule (i.e. "function") doesn't get called at all, anywhere, then are any targets declared within that rule added to the global container, or does the rule have to be called to add the targets? Or to put it another way, can you use rules to govern what targets end up being declared? — davidA, Oct 11 '12 at 21:08
If a rule doesn't get called at all, the code in its body doesn't get executed, hence, naturally, no target declarations that the body contains come into live. Those targets never get actually declared, added to the global container. Neither their names are introduced as target names. If you explicitly request such a target name on command line to be built, you'll get an error like "don't know how to make X". So yes, you can use rules to govern what targets end up being declared. — usta, Oct 14 '12 at 17:30

Understanding bjam's targets, and how to specify new ones?

1 Answers1