Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-50548

Shared Library should be allowed to declare reusable stages

      Currently, we have a lot of reusable steps we use in our Jenkinsfiles like so:

      pipeline {
        stages {
          stage('Build') {
            steps {
              reusableBuild()
            }
          }
          stage('Test') {
            steps {
              reusableTest()
            }
          }
        } 
      }

      I feel it is very repetitive to always replicate the stage definition - the way I look at this, these should be reusable as well. A pipeline would then simply consist of reusable, composable stages:

      pipeline {
        stages {
          reusableBuild()
          reusableTest()
          stage 'Something individual' {
            steps {
              echo 'only for this project'
            }
          }
        } 
      }

      Maybe this is already possible, but at the moment I have no idea how to define reusable stages in a shared library - any hint would be very much appreciated. I would also be willing to provide a PR, if only I had an idea which code to touch

          [JENKINS-50548] Shared Library should be allowed to declare reusable stages

          Jesus Alvarez added a comment -

          I spent all day trying do this with Declarative Pipelines and finally had to give up. It was so easy to do things like this with Scripted Pipelines, but Declarative is the way going forward. It would be nice to have this with the shared pipeline library.

          Jesus Alvarez added a comment - I spent all day trying do this with Declarative Pipelines and finally had to give up. It was so easy to do things like this with Scripted Pipelines, but Declarative is the way going forward. It would be nice to have this with the shared pipeline library.

          Marcelo Luiz Onhate added a comment - - edited

          Hello,

          As a suggestion you can use the (not well documented) feature that loads scripts on the fly.

          #!groovy
          script {
              library identifier: "setup@$BRANCH_NAME", retriever: legacySCM(scm)
          }
          pipeline {
              stages {
                  stage('Setup & Deploy') {
                      steps {
                          setup()
                          deploy()
                      }
                  }
              }
          }
          

          this will load the libs from the same repository and branch you are checking out, it will go to folder /vars/ and load all .groovy files.

          library identifier: "setup@$BRANCH_NAME", retriever: legacySCM(scm)
          
          <ROOT>/
              - vars/
                  - deploy.groovy
                  - setup.groovy
          

          The name of the .groovy script will become a step on pipeline, as you can see the steps setup and deploy.

          setup.groovy

          #!/usr/bin/env groovy
          def call() {
              sh "echo 'YEAH'"
              sh "some/other/script.sh"
          }
          

          deploy.groovy

          #!/usr/bin/env groovy
          def call() {
              def props = readProperties file: 'environment.env'
              env.PROP_1 = props['PROP_1']
          
              sh "script/that/uses/PROP_1"
          }
          

           

          Marcelo Luiz Onhate added a comment - - edited Hello, As a suggestion you can use the (not well documented) feature that loads scripts on the fly. #!groovy script { library identifier: "setup@$BRANCH_NAME" , retriever: legacySCM(scm) } pipeline { stages { stage( 'Setup & Deploy' ) { steps { setup() deploy() } } } } this will load the libs from the same repository and branch you are checking out, it will go to folder /vars/ and load all .groovy files. library identifier: "setup@$BRANCH_NAME", retriever: legacySCM(scm) <ROOT>/ - vars/ - deploy.groovy - setup.groovy The name of the .groovy script will become a step on pipeline, as you can see the steps setup and deploy . setup.groovy #!/usr/bin/env groovy def call() { sh "echo 'YEAH' " sh "some/other/script.sh" } deploy.groovy #!/usr/bin/env groovy def call() { def props = readProperties file: 'environment.env' env.PROP_1 = props[ 'PROP_1' ] sh "script/that/uses/PROP_1" }  

          Steven Foster added a comment -

          Declarative stages have become so flexible now: they can have their own options, environment, post stages, agent etc.

          This makes the ability to reuse and share them even more powerful. My use case is running tests against a selection of specialised hardware, accessible by a generic build agent. In conjunction with the lockable resources plugin, the typical stage for this in the middle of a pipeline looks something like:

          stage ('Run tests on kit') {
            agent { label 'local-kits' }
            options {
              lock(label: 'kit-type-1', variable: 'KIT_IP', quantity: 1)
            }
            steps {
              runTest('example-test.zip') // existing shared function to download and run previously archived binary on reserved kit
            }
            post {
              always {
                junit 'results.xml'
              }
            }
          }

          I use this in a ton of places, and help others include it in their pipelines. If I could replace all that (or maybe just the contents of stage { } ) with a call to a library, it would be really handy.

          Steven Foster added a comment - Declarative stages have become so flexible now: they can have their own options, environment, post stages, agent etc. This makes the ability to reuse and share them even more powerful. My use case is running tests against a selection of specialised hardware, accessible by a generic build agent. In conjunction with the lockable resources plugin, the typical stage for this in the middle of a pipeline looks something like: stage ( 'Run tests on kit' ) { agent { label 'local-kits' } options { lock(label: 'kit-type-1' , variable: 'KIT_IP' , quantity: 1) } steps { runTest( 'example-test.zip' ) // existing shared function to download and run previously archived binary on reserved kit } post { always { junit 'results.xml' } } } I use this in a ton of places, and help others include it in their pipelines. If I could replace all that (or maybe just the contents of stage { } ) with a call to a library, it would be really handy.

          Tom Ghyselinck added a comment - - edited

          Hi abayer,

          Is this really a duplicate of JENKINS-49135?

          In PR #241, you explicitly say that you won't support the directives stage, stages, steps, etc.
          and this is actually what we do want here.

          With best regards,
          Tom.

          Tom Ghyselinck added a comment - - edited Hi abayer , Is this really a duplicate of JENKINS-49135 ? In PR #241 , you explicitly say that you won't support the directives stage , stages , steps , etc. and this is actually what we do want here. With best regards, Tom.

          Frank Gen added a comment -

          Hello,

          This fix is marked as "Fixed but Unreleased", can someone add a pointer to the PR implementing this? Or any documentation?

          Thanks

          • Frank

          Frank Gen added a comment - Hello, This fix is marked as "Fixed but Unreleased", can someone add a pointer to the PR implementing this? Or any documentation? Thanks Frank

          Colm O'Shea added a comment -

          Would also love to see if there is docs or something for this?

          Colm O'Shea added a comment - Would also love to see if there is docs or something for this?

          CJ Harmath added a comment - - edited

          I would like this as well.

          Use case:

          share build stages across similar projects, to keep things DRY and have a central shared library defining the various build stages per project types.

          i.e.: I might have many many microservices all using the same 1) package restore 2) lint 3) build 4) run unit tests  stages with only parameter differences and it would be awesome if app devs can just include a single line with the necessary parameters to call these 4 steps.

           

          so instead of

           

          stages {
              stage('restore') {
               steps {
                echo 'package restore'      
                restore()     
               }
          
            }
          stage('build') {
             steps {
               echo 'building'
               build()
              }
          
            }
          stage('test') {
              steps {
                echo 'testing'
                test()
               }
          
            }
           }
          

           

          you could do this:

           

          stages {   
             sharedMicroserviceStages()
          }
          

           

           

          CJ Harmath added a comment - - edited I would like this as well. Use case: share build stages across similar projects, to keep things DRY and have a central shared library defining the various build stages per project types. i.e.: I might have many many microservices all using the same 1) package restore 2) lint 3) build 4) run unit tests  stages with only parameter differences and it would be awesome if app devs can just include a single line with the necessary parameters to call these 4 steps.   so instead of   stages {    stage( 'restore' ) {     steps { echo ' package restore'      restore()     }   } stage( 'build' ) {   steps { echo 'building' build() }   } stage( 'test' ) {    steps { echo 'testing'      test() }  } }   you could do this:   stages {       sharedMicroserviceStages() }    

          CJ Harmath added a comment -

          Okay, so I just re-read the docs and as I understand what I should be doing is to just put the entire pipeline into my shared library.

          https://jenkins.io/doc/book/pipeline/shared-libraries/#defining-declarative-pipelines

          This kind of makes me wonder if we allow defining entire pipeline, why not allow stages as well ?

          But I guess I am already happy with this as I only need to maintain one pipeline per app type.

          CJ Harmath added a comment - Okay, so I just re-read the docs and as I understand what I should be doing is to just put the entire pipeline into my shared library. https://jenkins.io/doc/book/pipeline/shared-libraries/#defining-declarative-pipelines This kind of makes me wonder if we allow defining entire pipeline, why not allow stages as well ? But I guess I am already happy with this as I only need to maintain one pipeline per app type.

          Dan Alvizu added a comment -

          For anyone else who runs into this: Andrew closed this as 'FIxed but unreleased' but with a resolution of 'Duplicate'.

           

          The duplicate issue is JENKINS-49135 and track there

          Dan Alvizu added a comment - For anyone else who runs into this: Andrew closed this as 'FIxed but unreleased' but with a resolution of 'Duplicate'.   The duplicate issue is  JENKINS-49135 and track there

            Unassigned Unassigned
            tobilarscheid Tobias Larscheid
            Votes:
            22 Vote for this issue
            Watchers:
            51 Start watching this issue

              Created:
              Updated:
              Resolved: