5

I would like to launch multiple ninja builds simultaneously. Each build is in its own directory and has its own build.ninja file.

I could just do the following:

cd <build-dir-1>
ninja &
cd <build-dir-2>
ninja &
...
cd <build-dir-last>
ninja

....but there are a couple of issues with this:

  • The default number of threads used by Ninja probably isn't optimal when launching multiple independent builds simultaneously.
  • Output will, I expect, be interleaved in a non-sensible way.

EDIT I could also just keep the ninja calls in the foreground (which is what I'm currently doing), but then there would be no easy way to estimate what the current progress of the (entire) build is.

So, I would like to do one of the following:

  1. merge the build.ninja files into one big file that can perform both builds in one ninja invocation.
  2. somehow launch ninja with multiple target build.ninja scripts.

It doesn't look like that second option is supported by ninja, but the first seems like it could be done easily enough using subninja <build-dir-n>/build.ninja. Has anyone done something like this before? Are there any hidden pitfalls? Alternatively, I could just perform the builds in sequence (i.e. the above sequence but without the &s), but this doesn't feel like the right solution.

Use-case

I'm using CMake, which generates a separate build.ninja file for each build configuration (release and debug). I'm also targeting multiple platforms, so I have multiple compiler, and CMake must be run separately for each platform. So if I want to build release and debug code for all platforms, I need to run ninja multiple times.

Kyle Strand
  • 15,941
  • 8
  • 72
  • 167
  • 1
    Why don't you simply build sequentially your release and debug version and your multiple platform, each one with the optimal number of threads? `cd && ninja && cd && ninja && cd [...]` – Antonio Aug 04 '15 at 09:34
  • @Antonio That's what I'm currently doing, but I was hoping there might be a better way. – Kyle Strand Aug 04 '15 at 15:44
  • 1
    Are you speaking of builds that involve a lot of files? In this case I would struggle to see a benefit. Do you have to trigger all these builds at each file modifications (therefore trigger a build on a few files), or the full set of builds is only seldom necessary? – Antonio Aug 04 '15 at 19:51
  • Use GNU Parallel to build them all in parallel: `find . -depth 1 -type d -print0 | parallel -0 cd {}'&&' ninja` – sakra Aug 04 '15 at 20:30
  • @sakra That's interesting (and I think your comment is how I learned about GNU Parallel, so thank you), but I expect it would interact poorly with Ninja's built-in parallelism smartness. – Kyle Strand Dec 22 '15 at 21:54
  • @Antonio I've edited the question to mention that one of the primary frustrations with running sequential foreground `ninja` operations is that estimating the build progress becomes impossible. – Kyle Strand Dec 22 '15 at 22:28
  • 3
    Imagine you do a full build, then make a single source change that means each ninja build needs to build, say, three files. There is no optimal number of threads, since each individual ninja run only can run three simultaneous compiles. A top-level ninja build could run many more in parallel. In our case, we have many ninja directories (created by cmake) for combinations of architecture/debug,release/etc. – James Moore Jan 30 '18 at 19:39

0 Answers0