0

I need to pick up in the make file the number of processor used for the paralell compilation.

e.g.

make -j32 .....

I need to pick up the number 32 inside the Makefile.

I know this comes inside the variable MAKEFLAG so I could parse it, but is there some other variable that gives this information directly?

For example:

NUMCPU = 32

DevSolar
  • 67,862
  • 21
  • 134
  • 209
Mariano
  • 240
  • 3
  • 15
  • 1
    ...why? If your Makefile "has to" react to the level of parallelization, chances are you're trying to solve a problem further up the pipe, which is likely to run afoul of the parallelization logic of `make` itself (like only counting *working* instances against the limit, not recursive ones). What is your *actual* problem? – DevSolar May 25 '22 at 06:44
  • because in the make file I need to use that information , the number of cpu, to give you more information in my makefile I do two main action 1) to compile some libraries with make 2) to compile other libraries with ninja, so inside the make file I have some phony to trigger ninja compilation, so I need to provide also to ninja the information of the number of processor to be used for the parallel computing – Mariano May 25 '22 at 06:58
  • 2
    That's **exactly** the kind of broken logic I was referring to. If one of your `make` rules is using `ninja` to do its job, that is still **one** worker thread. If you ran `make -j32`, and you take that `N := 32` for that `ninja` as well, you now have not 32, but 64 worker threads (32 `make`, 32 `ninja`). Parallelize on *one* level (make -> N = 32), not X levels (make, ninja, ... -> N = 32^X). -- Also, your project should always use *installed* versions of dependencies, build and installed by *those project's* build systems. Work on your project *or* its dependencies, not both at once. – DevSolar May 25 '22 at 08:29
  • 1
    Agree with @DevSolar but just to answer the question: no, there is no other variable that provides the level of parallelism. Only `MAKEFLAGS`. And note that in versions of GNU make older than 4.2, you can't even find it there. – MadScientist May 25 '22 at 21:39

2 Answers2

0

already solved in

GNU Make: Check number of parallel jobs

NUMPROC = $(patsubst -j%,%,$(filter -j%,$(MAKEFLAGS))

Mariano
  • 240
  • 3
  • 15
  • 1
    Note my comment above. If you are passing the level of parallelization given to `make` to subcommands (like `ninja` called from `make`), you are increasing the number of processes *exponentially*. You're looking at a broken build configuration. – DevSolar May 25 '22 at 09:45
  • no , I pass the number of job to a script that calls ninja and looking the process used with top cmd the number is correct – Mariano May 27 '22 at 10:00
  • Yet still you will end up with a `make` that runs `NUMPROC` working instances, calling a script calling `ninja` potentially calling `NUMPROC` instances in turn, so you end up with `NUMPROC * 2` working instances. Besides, `ninja` is guessing the number of CPU cores by default anyway... – DevSolar May 27 '22 at 10:03
  • @DevSolar ok so could I use .NOTPARALLEL for the target that will trigger the ninja build , but providing it the number of jobs to be used? – Mariano May 27 '22 at 10:15
  • I think the error is a bit further up the pipe. Why is there a mix of `make` and `ninja` in the first place? If I understood you correctly, the `ninja` build is for a stand-alone library you are depending on. In that case you should not trigger rebuilds of that library from your project at all, but rely on whatever is actually installed -- if the library changed, build (and install) it using its own (`ninja`) build system, *then* use your project's `make` to conditionally rebuild. Perhaps post a new question detailing your build structure and how to improve it. – DevSolar May 27 '22 at 10:26
0

@DevSolar me too would like not mix make and ninja build But I need to do,the project is a big project involving several libraries and several teams so I cannot decide alone the build process. in order to explain the build process I have a target and some libraries that use for the build system meson/ninja and other libraries that use make. Now during the official release phase all the library must be recompiled, so first are compiled the ones with the legacy "make " and then the ones with meson and the final the binary/executable that will link al of the previous compiled libraries.

At the moment all is triggered by a make command and the production team wants to use the -j option both for make and ninja. for this reason I am tring to provide the -j to the libraries and final binary/executable built with ninja.

Mariano
  • 240
  • 3
  • 15
  • This should be a new *question*, not an "answer" to this one. What I was refereing to earlier is that either a) each team should build / release its library independently, so everyone is using only what the team in question has **released**, or b) the whoe project should use one *integrated* build system / process (depending on how independent the libraries / teams are) => toss out either `make` or `ninja`, and build *one* build configuration for the whole. – DevSolar May 27 '22 at 10:40
  • Ok but tring do not talking about best practice, could I solve the problem I have with the following 2 steps: 1) retrinvig he process number "NUMPROC = $(patsubst -j%,%,$(filter -j%,$(MAKEFLAGS))", 2) use" .NOTPARALLEL for the target that will trigger the ninja build" – Mariano May 27 '22 at 10:44
  • *sigh*... the issue then is that `make` rule -- building a whole sublibrary -- is running as a single thread, and will take a long time. There is no way around it, that build setup is broken and cannot be fixed by juggling a couple of command-line options. – DevSolar May 27 '22 at 11:09