Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-37515

RFE: input() blocks the executor in Declarative, add a stage attribute instead?

    • Icon: Improvement Improvement
    • Resolution: Duplicate
    • Icon: Major Major
    • None
    • Jenkins LTS 2.72 rpm on SLES 11 SP 4

      When using an input step in pipeline, the executor is blocked until you proceed or abort the input step.

      We have here the case that the code is build whenever it is changed in the scm via polling.
      Only a small number of the builds going to development, staging and production.
      As the executor is blocked, the maximum number of executors is reached in short time.

      The build pipeline plugin instead use an executor for every job it contains.

      Is it possible to break end the executor when input step is reached and restart after proceed answer?

          [JENKINS-37515] RFE: input() blocks the executor in Declarative, add a stage attribute instead?

          I think the issue will be with workspace. Executor will hold workspace that may be needed for continuation after input. Some adittional non-blocking input maybe needs to be created?

          Kanstantsin Shautsou added a comment - I think the issue will be with workspace. Executor will hold workspace that may be needed for continuation after input. Some adittional non-blocking input maybe needs to be created?

          Jesse Glick added a comment -

          If you need to release the executor, put input outside node. If you need to keep some files from the workspace, stash them before you leave.

          Jesse Glick added a comment - If you need to release the executor, put input outside node . If you need to keep some files from the workspace, stash them before you leave.

          Thanks for the tip jglick, that resolved it for me.

          Katherine Morgan added a comment - Thanks for the tip jglick , that resolved it for me.

          Torsten Kleiber added a comment - - edited

          In which version do you have tested this, does not work in 2.7.2 here with following example :

          {node

          { stage 'Build after Commit' } stage 'Deploy against developement' input 'Are you sure?'}

          Torsten Kleiber added a comment - - edited In which version do you have tested this, does not work in 2.7.2 here with following example : {node { stage 'Build after Commit' } stage 'Deploy against developement' input 'Are you sure?'}

          See my last comment.

          Torsten Kleiber added a comment - See my last comment.

          Hi @tkleiber,
          Your screenshot shows the job in the build history for your 'PipelineTest' job, not the 'Build Executor Status' list which is on the main jenkins page, see my screenshot.
          When I made the change suggested by Jesse, the jobs do drop out of the executor list.

          Katherine Morgan added a comment - Hi @tkleiber, Your screenshot shows the job in the build history for your 'PipelineTest' job, not the 'Build Executor Status' list which is on the main jenkins page, see my screenshot. When I made the change suggested by Jesse, the jobs do drop out of the executor list.

          This is a little bit confusing: the build history shows all the jobs running until you proceed or abort any job instande. Additional old "running" jobs are not deleted by log rotation. So how to batch delete old job instances or timeout these?

          Torsten Kleiber added a comment - This is a little bit confusing: the build history shows all the jobs running until you proceed or abort any job instande. Additional old "running" jobs are not deleted by log rotation. So how to batch delete old job instances or timeout these?

          Markus Till added a comment -

          Hi,
          I tried to create a job like this

          pipeline {
              *agent none*
              stages {
                   stage('Build') {
                  agent any
                      steps {
                          echo 'This is a minimal pipeline.'                
                      }
                  }
                    stage('Deploy') {
                        
                      *agent none*
                      steps {
                              input message: 'test', ok: 'test', submitterParameter: 'test'                
                      }
                  }
              }
          }
          

          I'll hoped that agent none for the deploy step would prevent me from a blocked agent but it doesn't
          I think there should be a way to create a pipeline inupt without blocking a whole worker thread. Are there any plans to do so?

          Markus Till added a comment - Hi, I tried to create a job like this pipeline { *agent none* stages { stage( 'Build' ) { agent any steps { echo 'This is a minimal pipeline.' } } stage( 'Deploy' ) { *agent none* steps { input message: 'test' , ok: 'test' , submitterParameter: 'test' } } } } I'll hoped that agent none for the deploy step would prevent me from a blocked agent but it doesn't I think there should be a way to create a pipeline inupt without blocking a whole worker thread. Are there any plans to do so?

          Jesse Glick added a comment -

          AFAIK that syntax in Declarative would work, but if it does not, that would be a bug report for pipeline-model-definition-plugin.

          Jesse Glick added a comment - AFAIK that syntax in Declarative would work, but if it does not, that would be a bug report for pipeline-model-definition-plugin .

          Jesse Glick added a comment -

          how to batch delete old job instances

          See https://github.com/jenkinsci/workflow-aggregator-plugin/tree/master/demo for milestone usage.

          Jesse Glick added a comment - how to batch delete old job instances See https://github.com/jenkinsci/workflow-aggregator-plugin/tree/master/demo for milestone usage.

          Rafael Pestano added a comment - - edited

          Hi guys,

          in proposed solution we need to use a 'none' agent at top level pipeline declaration. In this case we can't use post pipeline event like below due to --JENKINS-43578--:

           

          pipeline {
              *agent none*
              stages {
                  stage('Build') {
                  agent any
                      steps {
                          echo 'This is a minimal pipeline.'                
                      }
                  }
                    stage('Deploy') {
                        
                      *agent none*
                      steps {
                              input message: 'test', ok: 'test', submitterParameter: 'test'                
                      }
                  }
              }
          
            post {
              changed {
                  sendNotificationa(currentBuild.currentResult)
              }
             }
          } 

          How to have non blocking executor with user input in step AND post pipeline event?

           

          Thanks in advance.

          Rafael Pestano added a comment - - edited Hi guys, in proposed solution we need to use a 'none' agent at top level pipeline declaration. In this case we can't use post pipeline event like below due to -- JENKINS-43578 --:   pipeline { *agent none* stages { stage('Build') { agent any steps { echo 'This is a minimal pipeline.' } } stage('Deploy') { *agent none* steps { input message: 'test', ok: 'test', submitterParameter: 'test' } } } post { changed { sendNotificationa(currentBuild.currentResult) } } } How to have non blocking executor with user input in step AND post pipeline event?   Thanks in advance.

          Rafael Pestano added a comment - - edited

          Also I'd like to add that proposed solution have another caveat which is when you add agent none to the top level pipeline declaration and then declare agent any inside each stage. Now you need to use stash and unstash in all stages because the stage can run on a different agent, example: 

          pipeline {
              agent none
              
              stages {
                  stage('Build') {
                      agent any
          
                      steps {
                          sh 'mvn clean package'
                          stash includes: 'src/**, pom.xml, target/**', name: 'build'
                      }
                  }
                  stage('Aprove deployment?') {
                      agent none
          
                      steps {
                          input message: 'Aprove deployment?'   
                       }
                  }
          
                  stage('Deploy') {
                      agent any
          
                      steps {
                          unstash 'build'
                          echo 'Deploying artifacts generated in build stage'
                      }
                  }
              }
          
              post {
                  changed {
                      sendNotificationa(currentBuild.currentResult)
                  }
              }
          }  

          Please consider reopening this issue to allow agent none just inside a single stage instead of top level pipeline.   

          Rafael Pestano added a comment - - edited Also I'd like to add that proposed solution have another caveat which is when you add agent none  to the top level pipeline declaration and then declare agent any  inside each stage. Now you need to use stash and unstash  in all stages because the stage can run on a different agent, example:  pipeline { agent none stages { stage('Build') { agent any steps { sh 'mvn clean package' stash includes: 'src/**, pom.xml, target/**', name: 'build' } } stage('Aprove deployment?') { agent none steps { input message: 'Aprove deployment?' } } stage('Deploy') { agent any steps { unstash 'build' echo 'Deploying artifacts generated in build stage' } } } post { changed { sendNotificationa(currentBuild.currentResult) } } }   Please consider reopening this issue to allow agent none just inside a single stage instead of top level pipeline .   

          Rafael Pestano added a comment - - edited

          Also another usecase where proposed solution does'nt work well is workspace clean up, ex:

           

          pipeline {
              agent none
              
              stages {
                  stage('Build') {
                      agent any
          
                      steps {
                          sh 'mvn clean package'
                          stash includes: 'src/**, pom.xml, target/**', name: 'build'
                      }
                  }
                  stage('Aprove deployment?') {
                      agent none
          
                      steps {
                          input message: 'Aprove deployment?'   
                       }
                  }
          
                  stage('Deploy') {
                      agent any
          
                      steps {
                          unstash 'build'
                          echo 'Deploying artifacts generated in build stage'
                      }
                  }
              }
          
              post {
                  always {
                      cleanWs deleteDirs: true, patterns: [[pattern: '**/target', type: 'INCLUDE']] //won't work with agent none (Required context class hudson.FilePath is missing)
                  }
                  changed {
                      sendNotificationa(currentBuild.currentResult)
                  }
              }
          } 

           

          Also, even if post event worked with agent none we would still have issues because as each stage runs on it's own agent which workspace we are going to clean? 

          I think the proposed workaround fails on many cases that we must find a solution where we can declare agent none only on stages which require user input/manual approval. 

          Rafael Pestano added a comment - - edited Also another usecase where proposed solution does'nt work well is workspace clean up , ex:   pipeline { agent none stages { stage('Build') { agent any steps { sh 'mvn clean package' stash includes: 'src/**, pom.xml, target/**', name: 'build' } } stage('Aprove deployment?') { agent none steps { input message: 'Aprove deployment?' } } stage('Deploy') { agent any steps { unstash 'build' echo 'Deploying artifacts generated in build stage' } } } post { always { cleanWs deleteDirs: true, patterns: [[pattern: '**/target', type: 'INCLUDE']] //won't work with agent none (Required context class hudson.FilePath is missing) } changed { sendNotificationa(currentBuild.currentResult) } } }   Also, even if post event worked with agent none we would still have issues because as each stage runs on it's own agent which workspace we are going to clean?  I think the proposed workaround fails on many cases that we must find a solution where we can declare agent none only on stages which require user input/manual approval.  

          Rafael Pestano added a comment - - edited

          Reopening as proposed solution introduces issues with post event as explained in comments here, here and here

           I hope we can find another solution. 

          Rafael Pestano added a comment - - edited Reopening as proposed solution introduces issues with post event as explained in comments here , here  and here .   I hope we can find another solution. 

          Jesse Glick added a comment -

          rmpestano you seem to be discussing Declarative Pipeline. There is not currently a reasonable way to use input in Declarative that I know of. Would be an RFE in pipeline-model-definition-plugin (my preference would be for things like locking and input to be attributes of stages, rather than explicit steps). For Scripted Pipeline, just do not put input inside node.

          Jesse Glick added a comment - rmpestano you seem to be discussing Declarative Pipeline. There is not currently a reasonable way to use input in Declarative that I know of. Would be an RFE in pipeline-model-definition-plugin (my preference would be for things like locking and input to be attributes of stages, rather than explicit steps). For Scripted Pipeline, just do not put input inside node .

          Hi jglick, thank you very much for the clarification.

          Can we reopen this issue and just change the component to pipeline-model-definition-plugin? I think the title and comments describe very well the problem.

          Rafael Pestano added a comment - Hi jglick , thank you very much for the clarification. Can we reopen this issue and just change the component to pipeline-model-definition-plugin? I think the title and comments describe very well the problem.

          Oleg Nenashev added a comment -

          Reopened and recategorized the issue.

          Oleg Nenashev added a comment - Reopened and recategorized the issue.

          Jesse Glick added a comment -

          Seems like it was already implemented in JENKINS-48379.

          Jesse Glick added a comment - Seems like it was already implemented in  JENKINS-48379 .

          Liam Newman added a comment -

          Bulk closing resolved issues.

          Liam Newman added a comment - Bulk closing resolved issues.

            Unassigned Unassigned
            tkleiber Torsten Kleiber
            Votes:
            2 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated:
              Resolved: