Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-50548

Shared Library should be allowed to declare reusable stages

      Currently, we have a lot of reusable steps we use in our Jenkinsfiles like so:

      pipeline {
        stages {
          stage('Build') {
            steps {
              reusableBuild()
            }
          }
          stage('Test') {
            steps {
              reusableTest()
            }
          }
        } 
      }

      I feel it is very repetitive to always replicate the stage definition - the way I look at this, these should be reusable as well. A pipeline would then simply consist of reusable, composable stages:

      pipeline {
        stages {
          reusableBuild()
          reusableTest()
          stage 'Something individual' {
            steps {
              echo 'only for this project'
            }
          }
        } 
      }

      Maybe this is already possible, but at the moment I have no idea how to define reusable stages in a shared library - any hint would be very much appreciated. I would also be willing to provide a PR, if only I had an idea which code to touch

          [JENKINS-50548] Shared Library should be allowed to declare reusable stages

          Tobias Larscheid created issue -
          Tobias Larscheid made changes -
          Description Original: Currently, we have a lot of reusable steps we use in our Jenkinsfiles like so:
          {code:java}
          pipeline {
            stages {
              stage('Build') {
                steps {
                  reusableBuild()
                }
              }
              stage('Test') {
                steps {
                  reusableTest()
                }
              }
            }
          }{code}
          I feel it is very repetitive to always replicate the stage definition - the way I look at this, these should be reusable as well. A pipeline would then simply consist of reusable, composable stages:
          {code:java}
          pipeline {
            stages {
              reusableBuild()
              reusableTest()
              stage 'Something individual' {
                echo 'only for this project'
              }
            }
          }{code}
          Maybe this is already possible, but at the moment I have no idea how to define reusable stages in a shared library - any hint would be very much appreciated. I would also be willing to provide a PR, if only I had an idea which code to touch ;)
          New: Currently, we have a lot of reusable steps we use in our Jenkinsfiles like so:
          {code:java}
          pipeline {
            stages {
              stage('Build') {
                steps {
                  reusableBuild()
                }
              }
              stage('Test') {
                steps {
                  reusableTest()
                }
              }
            }
          }{code}
          I feel it is very repetitive to always replicate the stage definition - the way I look at this, these should be reusable as well. A pipeline would then simply consist of reusable, composable stages:
          {code:java}
          pipeline {
            stages {
              reusableBuild()
              reusableTest()
              stage 'Something individual' {
                steps {
                  echo 'only for this project'
                }
              }
            }
          }{code}
          Maybe this is already possible, but at the moment I have no idea how to define reusable stages in a shared library - any hint would be very much appreciated. I would also be willing to provide a PR, if only I had an idea which code to touch ;)
          Andrew Bayer made changes -
          Component/s New: pipeline-model-definition-plugin [ 21706 ]
          Component/s Original: pipeline [ 21692 ]

          Jesus Alvarez added a comment -

          I spent all day trying do this with Declarative Pipelines and finally had to give up. It was so easy to do things like this with Scripted Pipelines, but Declarative is the way going forward. It would be nice to have this with the shared pipeline library.

          Jesus Alvarez added a comment - I spent all day trying do this with Declarative Pipelines and finally had to give up. It was so easy to do things like this with Scripted Pipelines, but Declarative is the way going forward. It would be nice to have this with the shared pipeline library.

          Marcelo Luiz Onhate added a comment - - edited

          Hello,

          As a suggestion you can use the (not well documented) feature that loads scripts on the fly.

          #!groovy
          script {
              library identifier: "setup@$BRANCH_NAME", retriever: legacySCM(scm)
          }
          pipeline {
              stages {
                  stage('Setup & Deploy') {
                      steps {
                          setup()
                          deploy()
                      }
                  }
              }
          }
          

          this will load the libs from the same repository and branch you are checking out, it will go to folder /vars/ and load all .groovy files.

          library identifier: "setup@$BRANCH_NAME", retriever: legacySCM(scm)
          
          <ROOT>/
              - vars/
                  - deploy.groovy
                  - setup.groovy
          

          The name of the .groovy script will become a step on pipeline, as you can see the steps setup and deploy.

          setup.groovy

          #!/usr/bin/env groovy
          def call() {
              sh "echo 'YEAH'"
              sh "some/other/script.sh"
          }
          

          deploy.groovy

          #!/usr/bin/env groovy
          def call() {
              def props = readProperties file: 'environment.env'
              env.PROP_1 = props['PROP_1']
          
              sh "script/that/uses/PROP_1"
          }
          

           

          Marcelo Luiz Onhate added a comment - - edited Hello, As a suggestion you can use the (not well documented) feature that loads scripts on the fly. #!groovy script { library identifier: "setup@$BRANCH_NAME" , retriever: legacySCM(scm) } pipeline { stages { stage( 'Setup & Deploy' ) { steps { setup() deploy() } } } } this will load the libs from the same repository and branch you are checking out, it will go to folder /vars/ and load all .groovy files. library identifier: "setup@$BRANCH_NAME", retriever: legacySCM(scm) <ROOT>/ - vars/ - deploy.groovy - setup.groovy The name of the .groovy script will become a step on pipeline, as you can see the steps setup and deploy . setup.groovy #!/usr/bin/env groovy def call() { sh "echo 'YEAH' " sh "some/other/script.sh" } deploy.groovy #!/usr/bin/env groovy def call() { def props = readProperties file: 'environment.env' env.PROP_1 = props[ 'PROP_1' ] sh "script/that/uses/PROP_1" }  

          Steven Foster added a comment -

          Declarative stages have become so flexible now: they can have their own options, environment, post stages, agent etc.

          This makes the ability to reuse and share them even more powerful. My use case is running tests against a selection of specialised hardware, accessible by a generic build agent. In conjunction with the lockable resources plugin, the typical stage for this in the middle of a pipeline looks something like:

          stage ('Run tests on kit') {
            agent { label 'local-kits' }
            options {
              lock(label: 'kit-type-1', variable: 'KIT_IP', quantity: 1)
            }
            steps {
              runTest('example-test.zip') // existing shared function to download and run previously archived binary on reserved kit
            }
            post {
              always {
                junit 'results.xml'
              }
            }
          }

          I use this in a ton of places, and help others include it in their pipelines. If I could replace all that (or maybe just the contents of stage { } ) with a call to a library, it would be really handy.

          Steven Foster added a comment - Declarative stages have become so flexible now: they can have their own options, environment, post stages, agent etc. This makes the ability to reuse and share them even more powerful. My use case is running tests against a selection of specialised hardware, accessible by a generic build agent. In conjunction with the lockable resources plugin, the typical stage for this in the middle of a pipeline looks something like: stage ( 'Run tests on kit' ) { agent { label 'local-kits' } options { lock(label: 'kit-type-1' , variable: 'KIT_IP' , quantity: 1) } steps { runTest( 'example-test.zip' ) // existing shared function to download and run previously archived binary on reserved kit } post { always { junit 'results.xml' } } } I use this in a ton of places, and help others include it in their pipelines. If I could replace all that (or maybe just the contents of stage { } ) with a call to a library, it would be really handy.
          Andrew Bayer made changes -
          Link New: This issue duplicates JENKINS-49135 [ JENKINS-49135 ]
          Andrew Bayer made changes -
          Resolution New: Duplicate [ 3 ]
          Status Original: Open [ 1 ] New: Fixed but Unreleased [ 10203 ]

          Tom Ghyselinck added a comment - - edited

          Hi abayer,

          Is this really a duplicate of JENKINS-49135?

          In PR #241, you explicitly say that you won't support the directives stage, stages, steps, etc.
          and this is actually what we do want here.

          With best regards,
          Tom.

          Tom Ghyselinck added a comment - - edited Hi abayer , Is this really a duplicate of JENKINS-49135 ? In PR #241 , you explicitly say that you won't support the directives stage , stages , steps , etc. and this is actually what we do want here. With best regards, Tom.

          Frank Gen added a comment -

          Hello,

          This fix is marked as "Fixed but Unreleased", can someone add a pointer to the PR implementing this? Or any documentation?

          Thanks

          • Frank

          Frank Gen added a comment - Hello, This fix is marked as "Fixed but Unreleased", can someone add a pointer to the PR implementing this? Or any documentation? Thanks Frank

            Unassigned Unassigned
            tobilarscheid Tobias Larscheid
            Votes:
            22 Vote for this issue
            Watchers:
            51 Start watching this issue

              Created:
              Updated:
              Resolved: