5

Jenkins declarative pipeline is too powerful for us, often users can abuse it. We are thinking to use an opinionated YAML to describe CI/CD pipeline. And it seems there are two choices.

  1. Write a plugin and consume YAML and dynamically create stage / steps.
  2. Write a plugin to convert a YAML to Jenkins pipeline.

I am not expert on Jenkins, so I hope some expert can give some guidance and maybe an example.

Mig82
  • 4,856
  • 4
  • 40
  • 63
M L L
  • 51
  • 1
  • 3
  • You don't need to write plugin. You could create entire pipeline in shared library using `evaluate()`. [Example](https://stackoverflow.com/a/61108773/7571258). – zett42 May 08 '20 at 06:27
  • This plugin already exists due to a past GSOC project: https://www.jenkins.io/blog/2018/07/17/simple-pull-request-plugin/. – Matthew Schuchard May 08 '20 at 11:41
  • Good to know about those two solutions. – M L L May 09 '20 at 02:55
  • you shouldn't need to use evaluate. Using that would also prevent you to apply linting with Codenarc on the evaluated pipeline if you ever want to lint pipelines the future – fredericrous May 09 '20 at 22:40

4 Answers4

2

  1. using official plugin pipeline-as-yaml, but it has a fixed grammar.

  2. using or customization wolox-ci

  3. create your own shared libaray. However, they are easy from beginning but full grammer design is required when used widely. Here is a psudo code based on curry.

// create a file named yamlCompiler.groovy in shared library,
def call(str){
  def rawMap = readYaml(text: str)
  // consume yaml and get a lambda function
  return {
    stage{
      steps.each{it ->
        it."$type"(it)
      }
    }
  }
}

Use yamlCompiler in your jenkinsfile code block.

@Library('your libs name')
def str = 
'''
steps:
 - type: sh
   script: ls -la 
 - type: echo
   message: xxx
'''
Closure closure = yamlCompiler(str)
closure.call()
Miao1007
  • 944
  • 6
  • 22
1

I'm looking for a similar solution. We run hardened predefined pipelines for every project, but still want to allow dev teams to customise certain steps within the process —without allowing them the full power of a Jenkinsfile.

I'm also exploring the possibility of an —in your words— "opinionated YAML".

I've so far only found one example of such an implementation: Wolox-CI supports their own pre-defined build steps via YAML. You'll be able to see the steps they support here.

I'm thinking of parsing the YAML using Snake YAML. Here's an SO answer with an example on how to do it.

Mig82
  • 4,856
  • 4
  • 40
  • 63
0

Two solutions:

If you're not an expert and don't want/have the time to become one, the second solution might be the best one.

fredericrous
  • 2,833
  • 1
  • 25
  • 26
0

Really? Is the only difference here when the plugin is executed?:

  1. Write a plugin and consume YAML and dynamically create stage / steps.
  2. Write a plugin to convert a YAML to Jenkins pipeline.

Forgive me, because I may be a little hardened, but abstracting a layer for the dynamic creation of a declarative, or scripted, Jenkinsfile written in the simple groovy lang syntax so that it can be pretty-printed in yml prevents users from updating your yml exactly how? It seems to me your abstraction only adds to the complexity with which you wish to implement usability.

One, all the current yml plugins for Jenkins do exactly that. Two, they don't actually have the full breadth of "features" (yes, I'm using that term loosely here) accessible by implementing the groovy/(java) classes already available in the Jenkins domain (referencing the DSL). Two solutions exist right now for this, and I've investigated both, and implemented both, extensively. One is wolox-ci, which is the better of the two, and the other is Pipeline-as-YAML. In my opinion, it's easy to use, but both lack the full breadth of implementation features simply using groovy provides. So why force it? Simply so your users can have a pretty-printed yml file, and not have to be concerned with simple syntax, which you claim hardens your infrastructure-as-code backend so that the same users can't screw it up? Sorry, I'm calling bull pucky on that assertion. What's to stop anyone from totally screwing up your builds by pushing a change to the yml file which breaks the integration with groovy, or worse, completely changes an algorithm you worked hard to customize?

Sorry, I just don't get it. Sure, making something more human readable is always a good thing. Doing it because of the reasons you've stipulated makes no sense, though. Also, unless you have a super simple defined algorithm in your CI/CD process, without any non-continuous-passing-style transform methods being implemented, then using the current iterations of the yml-as-Jenkinsfile-templates plugins is probably not the way you want to go.

Now, you could write your own plugin to do this, but what's the technical debt on that, versus just learning the groovy syntax? Also, it still doesn't prevent users from making code changes to your build infrastructure, then integrating those changes in a simple yml file.

John Bain
  • 1