0

I want to load variably-named shared libraries in my Jenkinsfiles sequentially, and run a global variable method of the same name in each one.
I.e. as pseudocode, what I want to do is:

for lib in in [foo, bar]:
  load shared library(lib)
  run the shared library's global variable method named 'func()'

I tried implementing this as follows:

// Jenkinsfile
pipeline {
  agent any

    stages {
      stage('1') {
        steps {
          testLoadLib()
        }
      }
    }
}

void testLoadLib() {
  runGlobalVariableMethod('foo@dev')
  runGlobalVariableMethod('bar@dev')
}

void runGlobalVariableMethod(String libraryName) {
  def lib = library(libraryName)
  def d = [:]
  func.func(d)
}

// shared_lib_foo
// vars/func.groovy

def func(d) {
  println("foo func.groovy:func()")
}

// shared_lib_bar
// vars/func.groovy

def func(d) {
  println("bar func.groovy:func()")
}

The shared libraries have been configured as Jenkins Global Pipeline Libraries as follows: enter image description here

enter image description here

On running the Jenkins pipeline, func.func() from the foo@dev shared library is run twice, instead of from foo@dev once, then bar@dev next.
I.e.:
actual output (edited for brevity):

foo func.groovy:func()
foo func.groovy:func()

desired output:

foo func.groovy:func()
bar func.groovy:func()

Question: between the Jenkinsfile and the Jenkins shared libraries, how can I implement the desired behavior?
The desired behavior being: I want to load multiple shared libraries, iteratively, and run each one's global variable method (that has the same name).
One might describe this as a plugin architecture: load a variable/specifiable shared library, and run an expected function within it.

The unedited Jenkins pipeline output (for better context) shows that it appears to be checking out the correct shared libraries -- I see correct references to the respective shared libraries' git repository and commit-ids:

[Pipeline] // stage
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (1)
[Pipeline] library
Loading library foo@dev
Attempting to resolve dev from remote references...
 > /usr/bin/git --version # timeout=10
 > git --version # 'git version 2.20.1'
using GIT_ASKPASS to set credentials 
 > /usr/bin/git ls-remote -h -- ssh://git@bitbucket.company.com:8999/prj/foo.git # timeout=10
Found match: refs/heads/dev revision 12419a423b1da2827215bca89aaa0f1fdba7e6ae
The recommended git tool is: NONE
using credential f3d14e3a-851b-4837-bbfd-31292f16e310
 > /usr/bin/git rev-parse --resolve-git-dir /home/user/workspace/wip@libs/006019fd942f0d37b69f2cd051be759efa34f738b3937627946801f48dd03a7c/.git # timeout=10
Fetching changes from the remote Git repository
 > /usr/bin/git config remote.origin.url ssh://git@bitbucket.company.com:8999/prj/foo.git # timeout=10
Fetching without tags
Fetching upstream changes from ssh://git@bitbucket.company.com:8999/prj/foo.git
 > /usr/bin/git --version # timeout=10
 > git --version # 'git version 2.20.1'
using GIT_ASKPASS to set credentials 
 > /usr/bin/git fetch --no-tags --force --progress -- ssh://git@bitbucket.company.com:8999/prj/foo.git +refs/heads/*:refs/remotes/origin/* # timeout=10
Checking out Revision 12419a423b1da2827215bca89aaa0f1fdba7e6ae (dev)
 > /usr/bin/git config core.sparsecheckout # timeout=10
 > /usr/bin/git checkout -f 12419a423b1da2827215bca89aaa0f1fdba7e6ae # timeout=10
Commit message: "wip"
 > /usr/bin/git rev-list --no-walk 12419a423b1da2827215bca89aaa0f1fdba7e6ae # timeout=10
[Pipeline] echo
foo func.groovy:func()
[Pipeline] library
Loading library bar@dev
Attempting to resolve dev from remote references...
 > /usr/bin/git --version # timeout=10
 > git --version # 'git version 2.20.1'
using GIT_ASKPASS to set credentials 
 > /usr/bin/git ls-remote -h -- ssh://git@bitbucket.company.com:8999/prj/bar.git # timeout=10
Found match: refs/heads/dev revision 52bc665d148fd2109831f484e604f1e3f82e895b
The recommended git tool is: NONE
using credential f3d14e3a-851b-4837-bbfd-31292f16e310
 > /usr/bin/git rev-parse --resolve-git-dir /home/user/workspace/wip@libs/87e71759ef01c569f052b9dc73ff630019f50862832bd0583331e87472373144/.git # timeout=10
Fetching changes from the remote Git repository
 > /usr/bin/git config remote.origin.url ssh://git@bitbucket.company.com:8999/prj/bar.git # timeout=10
Fetching without tags
Fetching upstream changes from ssh://git@bitbucket.company.com:8999/prj/bar.git
 > /usr/bin/git --version # timeout=10
 > git --version # 'git version 2.20.1'
using GIT_ASKPASS to set credentials 
 > /usr/bin/git fetch --no-tags --force --progress -- ssh://git@bitbucket.company.com:8999/prj/bar.git +refs/heads/*:refs/remotes/origin/* # timeout=10
Checking out Revision 52bc665d148fd2109831f484e604f1e3f82e895b (dev)
 > /usr/bin/git config core.sparsecheckout # timeout=10
 > /usr/bin/git checkout -f 52bc665d148fd2109831f484e604f1e3f82e895b # timeout=10
Commit message: "wip"
 > /usr/bin/git rev-list --no-walk 52bc665d148fd2109831f484e604f1e3f82e895b # timeout=10
[Pipeline] echo
foo func.groovy:func()
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
StoneThrow
  • 5,314
  • 4
  • 44
  • 86

1 Answers1

0

tl;dr: it appears that:

  • The return-value of library (loads a shared library) isn't an object on which you can invoke the shared library's global variable methods.
  • The return-value of load (load a "local" Groovy script) is an object on which you can invoke the Groovy script's global variable methods.

Use load to load Groovy scripts from the local filesystem instead of library to load from source control.

The ugly details: the closest I've been able to achieve to an answer to the question asked is to use the load function, instead of library.

library loads groovy scripts directly from SCM, which may or may not be in the same repo as the Jenkinsfile itself.

load seems specifically only able to load from the local filesystem, wherever the Jenkinsfile exists.

Critical to this question, though, library seems unwilling to override global variable functions if newly-loaded shared libraries have groovy scripts with the same filename and function names. Also, library seems to not return any kind of object to which its functions are scoped.

This shortcoming appears to be absent with the load function: it returns an object to which the loaded groovy script's functions are scoped. This behavior is in line with lazy-loading shared libraries, e.g. with C's dlopen and dlsym functions.

This is perhaps better illustrated by example:

// Jenkinsfile
pipeline {
  agent any

  stages {
    stage('1') {
      steps {
        testLibrary()
      }
    }
  }
}

def testLibrary() {
  def repo_prefix = "ssh://git@bitbucket.company.com:8999/prj/"
  def repo_suffix = ".git"
  def branch = "@dev"
  def cred_id = 'ssh_credz'
  def lib_names = ['foo', 'bar']

  for (String lib_name in lib_names) {
    def lib = library(identifier: lib_name + branch, retriever: modernSCM(
        [$class: 'GitSCMSource',
         remote: repo_prefix + lib_name + repo_suffix,
         credentialsId: credz,
         ]))
    // lib.func(42) // java.lang.ClassNotFoundException: null
    func.func(42) // Not useful because global variable `func` is set
                  // in the first iteration of the for-loop but not
                  // overridden by subsequent iterations.
  }
}

The relevant output of running this Jenkins pipeline, as noted in the original post, is:

Loading library foo@dev
...
Found match: refs/heads/dev revision 288d0f105fa21be1ea8c0245c4dddcf16a5340c9
...
foo func.groovy:func()
...
Loading library bar@dev
...
Found match: refs/heads/dev revision 989b86211ff225b0d576b854d959b5da6ef641af
...
foo func.groovy:func()

This may be compared with equivalent-intent use of the load function:

// Jenkinsfile
pipeline {
  agent any

  stages {
    stage('1') {
      steps {
        testLoad()
      }
    }
  }
}

def testLoad() {
  def lib_names = ['foo', 'bar']

  for (String lib_name in lib_names) {
    stage(lib_name) {
      def lib_file = lib_name + '.groovy'
      def lib = load lib_file

      lib.func(42);
    }
  }
}

Running this Jenkins pipeline reveals the desired output:

foo func.groovy:func()
bar func.groovy:func()

In terms of implementation, the relevant differences are:

  1. To use load, the groovy script filenames need to be different, since you're loading by filename.
  2. The groovy scripts to be loaded must be in the same filesystem as the Jenkinsfile. I've found it convenient to simply include the .groovy files in the same folder and git repo as the Jenkinsfile.

An arguable demerit of this approach is that the loaded groovy script isn't as "decoupled" from the Jenkins pipeline as a true shared library: e.g. the groovy scripts don't exist in a separate git repo, which could be modified and maintained independent from the Jenkinsfile's git repo.
But I think with the right structure and workflow, the detriments can be minimized.
And this nicely achieves the desired goal: I can successively load multiple groovy scripts and execute each one's same-named global variable function.

StoneThrow
  • 5,314
  • 4
  • 44
  • 86