1

I have a Jenkins pipeline to build, test and commit the code changes to a git repository. While the pipeline is same(common), on successful build and test the code changes may be committed to one of the three git repositories based on the input provided by the user at the time of triggering the job.

While i'm aware that there is a configuration in Jenkins pipeline to disallow concurrent builds (Do not allow concurrent builds), i do not want to use it. Reason is, let's say if the first job is running for git_repo1 and the second job is running for git_repo2, i dont want to stall the second job for git_repo2 by selecting the above option. Since two jobs are running for two different git repositories, they should be allowed to run concurrently.

Therefore, i have decided to create a lock for the Jenkins job based on my code i.e. to pause/halt/wait the second build if the first build is already running for the same git repo.

Sample code is as under:

#!/usr/bin/perl

my $repo_name = "git_repo1";
my $dir = "/my/local/path/$repo_name";
my $cmd = "mkdir $dir";

if (-d $dir) 
    {
        print "directory exists.. \n";
        while (-d $dir)
        {
            # wait until $dir is deleted by the job which is already in progress. Until that time this Jenkins job will wait for its deletion.
            sleep 10;
            print ".";
        }
    } 
else 
{
    print "directory does not exists..\n";
    system ($cmd);
    print "directory created..\n";
    # Current Jenkins job may proceed.
}

print "end of file.. \n";

Please suggest if this code is robust and will survive in all conditions.

Yash
  • 2,944
  • 7
  • 25
  • 43

1 Answers1

1

I would simply isolate the git push in its own "publish" stage.

Meaning everything before that "publish" stage can run concurrently.

But for the "publish" stage itself, you can use the Jenkins Lockable Resource plugin, and declare a lock (as in here) in order to prevent to pushes to the same repo to happen at the same time. That and using a milestone step.

stage('Publish') {
    lock(resource: "repo ${REPO_NAME}", inversePrecedence: true) {
      milestone 1
      sh "git push ..."
    }
}

So:

  • no perl dependency
  • no script maintainance
  • everything concurrent (even for the same repo)
  • except the push part.
VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250
  • Thanks @VonC. I will try this solution as well because, as you said, everything is concurrent in this solution except the push part. Can you please elaborate more on the `milestone` step. Does it mean that the second pipeline will wait at the milestone if first job's milestone is not yet executed? The documentation doesn't explain much on this. – Yash May 02 '20 at 13:30
  • @Yash Main justification: https://stackoverflow.com/a/40185898/6309: "abort any older pipeline executions, if a more recent build already reached the milestone (you don't want to let an older build that hangs longer in CI overwrite the deployment of a newer build, do you?)". See if it useful in your case. – VonC May 02 '20 at 13:35