Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-42224

Need ability to create reusable chunks of Declarative Pipeline

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      There is no way to write reusable chunks of Declarative Pipeline configuration.
      Jenkinsfile has no facility for this, and
      shared libraries support only scripted groovy (classes, steps, globals).

        Attachments

          Issue Links

            Activity

            Hide
            abayer Andrew Bayer added a comment -

            Nico Navarrete - your particular use case is exactly what I'm now experimenting with over in JENKINS-46547, FYI.

            Show
            abayer Andrew Bayer added a comment - Nico Navarrete - your particular use case is exactly what I'm now experimenting with over in JENKINS-46547 , FYI.
            Hide
            fxnn Felix Neumann added a comment - - edited

            Andrew Bayer: -JENKINS-46547- was a great step forward. From my point of view, the biggest remaining problem is that one often copy-and-pastes whole pipeline to have different variations, e.g. with/without a "Integration Test" stage, with/without special Post-Steps, with/without special tools etc.pp.

            Therefore I'd like to see some kind of composable pipelines, where one could have a basic

            pipeline("my-shared-pipeline") {
              agent { /* ... */ }
              tools { /* ... */ }
              options { /* ... */ }
              stages {
                stage('Compile and Test') { /* ... */ }
                stage('Deploy') { /* ... */ }
              }
              post {
                failure { /* ... */ }
                alway { /* ... */ }
              }
            }
            

            and then extend it, using some mechanism to make up the final order of stages:

            pipeline(extends: "my-shared-pipeline") {
              stages {
                stage('Integration Test', after: 'Compile and Test') { /* ... */ }
              }
            }
            

            This way one would neither need to copy-and-paste the whole pipeline, nor need to introduce empty, never executed stages into builds.

            Btw., loading chunks from resources (as proposed above) also sounds like a good solution.

            Just my 5 Cents – thanks for making all this possible!

            Show
            fxnn Felix Neumann added a comment - - edited Andrew Bayer : - JENKINS-46547 - was a great step forward. From my point of view, the biggest remaining problem is that one often copy-and-pastes whole pipeline to have different variations, e.g. with/without a "Integration Test" stage, with/without special Post-Steps, with/without special tools etc.pp. Therefore I'd like to see some kind of composable pipelines, where one could have a basic pipeline( "my-shared-pipeline" ) { agent { /* ... */ } tools { /* ... */ } options { /* ... */ } stages { stage( 'Compile and Test' ) { /* ... */ } stage( 'Deploy' ) { /* ... */ } } post { failure { /* ... */ } alway { /* ... */ } } } and then extend it, using some mechanism to make up the final order of stages: pipeline( extends : "my-shared-pipeline" ) { stages { stage( 'Integration Test' , after: 'Compile and Test' ) { /* ... */ } } } This way one would neither need to copy-and-paste the whole pipeline, nor need to introduce empty, never executed stages into builds. Btw., loading chunks from resources (as proposed above) also sounds like a good solution. Just my 5 Cents – thanks for making all this possible!
            Hide
            tobilarscheid Tobias Larscheid added a comment -

            I just opened https://issues.jenkins-ci.org/browse/JENKINS-50548 for defining reusable stages, maybe it is interesting for you as well.

            Show
            tobilarscheid Tobias Larscheid added a comment - I just opened https://issues.jenkins-ci.org/browse/JENKINS-50548  for defining reusable stages, maybe it is interesting for you as well.
            Hide
            nroose Nick Roosevelt added a comment -

            I would like this as well. But also, it seems like we can make some of those blocks one line, and we could use a preprocessor, if necessary. Just starting to get into this, but seems like groovy should support this already.

            Show
            nroose Nick Roosevelt added a comment - I would like this as well. But also, it seems like we can make some of those blocks one line, and we could use a preprocessor, if necessary. Just starting to get into this, but seems like groovy should support this already.
            Hide
            bassam_khouri Bassam Khouri added a comment -

            I would like this as well. In my case, I have a single pipeline with multiple parallel stages. Each parallel stages have 90%+ of identical code and I would like to programatically defines different stages.

            for example, my pipeline looks like this

            pipeline {
                stages {
                    stage("stage1") {
                        agent {
                            // different agent based on stage
                        }
                        environment (
                            BUILD_OPTIONS = "different build options per stage"
                        )
                        steps {
                            deleteDir()
                            echo 'Building..'
                            build(BUILD_OPTIONS)
                        }
                        post {
                            // has some slight differences here depending on the stage
                        }
                    }
                    stage("stage2") {
                        agent {
                            // different agent based on stage
                        }
                        environment (
                            BUILD_OPTIONS = "another  build options per stage"
                        )
                        steps {
                            deleteDir()
                            echo 'Building..'
                            build(BUILD_OPTIONS)
                        }
                        post {
                            // has some slight differences here depending on the stage
                        }
                    }
                    ...
                    stage("stageN") {
                        agent {
                            // different agent based on stage
                        }
                        environment (
                            BUILD_OPTIONS = "build options for stage N"
                        )
                        steps {
                            deleteDir()
                            echo 'Building..'
                            build(BUILD_OPTIONS)
                        }
                        post {
                            // has some slight differences here depending on the stage
                        }
                    }
            
                }
            }
            

            Personally, I would like to do something like this

            def myStage(Map args) {
                // define the "dynamic" stage that uses the 'args' parameter
                if (args.specialAgent) {
                    agent { docker {...} }
                } else {
                    agent { label "SpecialAgent" } 
                }
                environment {
                   BUILD_OPTIONS = " ".join(args.buildOptions)
                }
                steps {
                   ...
                }
                post {
                    always {
                        if {args.UploadLogs) {
                            // call an "action"
                        }
                        // more actions
                    }
                }
            }
            
            pipeline {
                stages {
                    stage1 myStage([buildOptions: ["build", "option", "stage1"], arg1: "something"])
                    stage2 myStage([buildOptions: ["stage2"], arg1: "something"])
                    ...
                    stageN myStage([buildOptions: [STAGE_NAME], arg1: "something"0)
                }
            }
            
            Show
            bassam_khouri Bassam Khouri added a comment - I would like this as well. In my case, I have a single pipeline with multiple parallel stages. Each parallel stages have 90%+ of identical code and I would like to programatically defines different stages. for example, my pipeline looks like this pipeline { stages { stage( "stage1" ) { agent { // different agent based on stage } environment ( BUILD_OPTIONS = "different build options per stage" ) steps { deleteDir() echo 'Building..' build(BUILD_OPTIONS) } post { // has some slight differences here depending on the stage } } stage( "stage2" ) { agent { // different agent based on stage } environment ( BUILD_OPTIONS = "another build options per stage" ) steps { deleteDir() echo 'Building..' build(BUILD_OPTIONS) } post { // has some slight differences here depending on the stage } } ... stage( "stageN" ) { agent { // different agent based on stage } environment ( BUILD_OPTIONS = "build options for stage N" ) steps { deleteDir() echo 'Building..' build(BUILD_OPTIONS) } post { // has some slight differences here depending on the stage } } } } Personally, I would like to do something like this def myStage(Map args) { // define the "dynamic" stage that uses the 'args' parameter if (args.specialAgent) { agent { docker {...} } } else { agent { label "SpecialAgent" } } environment { BUILD_OPTIONS = " " .join(args.buildOptions) } steps { ... } post { always { if {args.UploadLogs) { // call an "action" } // more actions } } } pipeline { stages { stage1 myStage([buildOptions: [ "build" , "option" , "stage1" ], arg1: "something" ]) stage2 myStage([buildOptions: [ "stage2" ], arg1: "something" ]) ... stageN myStage([buildOptions: [STAGE_NAME], arg1: "something" 0) } }

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              bitwiseman Liam Newman
              Votes:
              79 Vote for this issue
              Watchers:
              92 Start watching this issue

                Dates

                Created:
                Updated: