1

I have a Jenkins pipeline that has 6 stages.

In 4 of the 6 stages the pipeline triggers Jenkins jobs using build job.

Both the pipeline and the job are set to allow concurrent parallel executions.

The node on which the pipeline and the jobs run is a single node having number of executer set to 10.

Everything works fine when I run 10 parallel concurrent pipeline.

But if I run more than 10 parallel pipelines then all of them seem to go in a deadlock, none of them completes no matter how long you wait and send they are waiting on each other to complete.

If I kill the 11th execution then all the 10 start completing successfully.

My requirement is that if someone executes more concurrent builds of a pipeline than the number of executors it has for the node that it runs on; then in that case 10 should complete parallel execution and 11 onwards should wait until then and should be executed in the second batch of 10 executions rather than all going to a hung state.

Please help me understand if this is a bug with Jenkins latest version and what is the workaround to avoid all pipeline builds from falling in hung state?

mohtashim
  • 57
  • 1
  • 8
  • 1
    check this https://github.com/jenkinsci/throttle-concurrent-builds-plugin – Yuri G. Feb 15 '20 at 08:45
  • I'm not sure I completely understand what you are trying to achieve. Could you add a [mcve]? Do the pipelines that launch other jobs really have to wait for the jobs to finish? Otherwise pass `false` for the `wait` parameter of the `build` step, which allows the current pipeline to end, freeing a slot for the launched job. Another way without the use of plugins: create a dedicated node (possible even on the same machine) for the launched jobs, so they don't have to fight with their launchers for executors. – zett42 Feb 15 '20 at 18:36

1 Answers1

0

The issue can be that the master or the node runs out of CPU and/or memory. You can also look at Jenkins master/slave node logs for exceptions.

K. B.
  • 3,342
  • 3
  • 19
  • 32