3

Is there a way to get current job number to use in makefile rule?

Let me give you a little context. I am using a tool which runs on multiple files. Naturally I use parallel jobs to speed things up. The real issue here is that this tool spawns multiple threads and I would like them to run in single core - since that way it is faster. I did some tests and it runs faster that way.

I need job numer to set process affinity to cpu core.

Wojciech
  • 330
  • 2
  • 13
  • If you make version is above 4.2, check possbile answer:https://stackoverflow.com/questions/48863762/gnu-make-check-number-of-parallel-jobs – Yingchun Nov 20 '21 at 02:31

1 Answers1

1

There is no way to get a "job number" because make doesn't track a job number. Basically all the instances of make in a single build share a list of identical tokens. When make wants to start a job it obtains a token. When make is finished with a job it adds the token back to the list. If it tries to get a token and one is not available it will sleep until one becomes available. There's no distinguishing characteristic to the tokens so there's no way to have a "job number".

To learn more about how GNU make handles parallel builds, you can read http://make.mad-scientist.net/jobserver.html

I'm not quite sure how this helps you anyway. Make doesn't know anything about threads, it only starts processes. If a single process consists of multiple threads, make will still think of it as a single job.

EDIT: Assuming that you are in a single, non-recursive invocation of make you can do it like this:

COUNT :=

%.foo :
        $(eval COUNT += x)
        @echo "$@: Number of rules run is $(words $(COUNT))"

all: a.foo b.foo c.foo d.foo
MadScientist
  • 92,819
  • 9
  • 109
  • 136
  • I was affraid of this. I've browsed makefile sourcecode and did not see anything relevant to my need. Maybe you have other suggestion - like how to get uniqe id for each tool run? I was thinking of something like $RANDOM - but how random is random ;) – Wojciech May 21 '13 at 12:51
  • I still don't understand why you need this, but why not just use the PID? That's a system-wide unique value. – MadScientist May 21 '13 at 13:35
  • Yes, PID is unique as you said. I did not mention that I needed consecutive unique IDs - so situation like 2 processes running on the same cpu core would not happen. – Wojciech Jun 18 '13 at 10:10
  • I know of no way to do this across a recursive build environment. If you are in a single, non-recursive makefile then you could do something like use `$(eval ...)` in the rule to modify a variable every time it was invoked. This would let you "count" them. – MadScientist Jun 18 '13 at 19:27
  • There is a very good case where one would need a job id: to extract the log of a job that failed. One could prefix the commands' output with job id, in order to untangle the all-interleaved output of `make`. `--output-sync=target` would help, but it's too slow, and still the error is not at the end of a log. Another case is to monitor jobs' execution as they run, and switch between the logs of a different jobs on the fly. – Victor Sergienko May 28 '19 at 23:19
  • Output sync does not (should not) slow down make. It may _appear_ as if it's slower since output is delayed until the command is finished. But the extra processing time for output sync should not be noticeable. For the kind of job ID you are talking about, the PID works fine. – MadScientist May 29 '19 at 14:03