Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-47286

Support skipping stages in scripted pipelines for nice visualization in blue ocean and classic UI stage view

      Created as response to comments of https://issues.jenkins-ci.org/browse/JENKINS-37781 as the visualization of skipped stages is very, very nice for declarative pipelines in Blue Ocean.

      1. Allow to skip stages in scripted pipelines leading to equally nice visualization in Blue Ocean: The current approaches mentioned by mkobit in https://issues.jenkins-ci.org/browse/JENKINS-37781?focusedCommentId=294965&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-294965 and jamesdumay in https://issues.jenkins-ci.org/browse/JENKINS-37781?focusedCommentId=294966&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-294966 lead to either misleading or just less obvious visualization...
      2. Improve visualization of stage view: instead of showing skipped stages (declarative pipelines) as always being green and allegedly executed, make them e.g. gray.

          [JENKINS-47286] Support skipping stages in scripted pipelines for nice visualization in blue ocean and classic UI stage view

          James Dumay added a comment -

          Hi there,
          As I mentioned on JENKINS-37781, we won't be adding the skipped feature to Pipeline Script. There are some features we intend that will remain to be Pipeline and Blue Ocean only.

          However, if someone from the community wants to pick this up and work on it, please reopen and assign this to yourself.

          Thanks,
          James

          James Dumay added a comment - Hi there, As I mentioned on JENKINS-37781 , we won't be adding the skipped feature to Pipeline Script. There are some features we intend that will remain to be Pipeline and Blue Ocean only. However, if someone from the community wants to pick this up and work on it, please reopen and assign this to yourself. Thanks, James

          Ben Dean added a comment -

          jamesdumay are scripted pipelines deprecated or something? Why is declarative prefered? I would really like to have the ability to use a when clause on stages in scripted pipeline, for the very same reasons people use it in declarative.

          Ben Dean added a comment - jamesdumay are scripted pipelines deprecated or something? Why is declarative prefered? I would really like to have the ability to use a when clause on stages in scripted pipeline, for the very same reasons people use it in declarative.

          Bill Hamilton added a comment -

          jdumay I want to throw in my support to this:
          1)The ability to skip a stage is very common in our organization. A repo with many branches might want to skip various stages (deployment, testing) for certain branches. They can obviously do nothing, but again this is misleading in the pipeline view since it looks like to performed the stage (just really really fast)
          2)This support needs to be in scripted as well as declarative. You guys know that scripted is alive and well right? And sometimes, the only option....

          Bill Hamilton added a comment - jdumay I want to throw in my support to this: 1)The ability to skip a stage is very common in our organization. A repo with many branches might want to skip various stages (deployment, testing) for certain branches. They can obviously do nothing, but again this is misleading in the pipeline view since it looks like to performed the stage (just really really fast) 2)This support needs to be in scripted as well as declarative. You guys know that scripted is alive and well right? And sometimes, the only option....

          Jordi Riera added a comment -

          Hello,

          Could you detail why the reasons the `skipped` feature would be only declarative? I always struggle with choosing between using declarative and scripted, but I most of the time end up with using scripted as our pipelines are pretty complex here. 

          I would be interested to see this feature in scripted for sure.

          Jordi Riera added a comment - Hello, Could you detail why the reasons the `skipped` feature would be only declarative? I always struggle with choosing between using declarative and scripted, but I most of the time end up with using scripted as our pipelines are pretty complex here.  I would be interested to see this feature in scripted for sure.

          Philip Stroh added a comment -

          I can just agree - from my perspective the ability to skip stages and visualize that stages are skipped is a (long awaited) must have feature for both declarative and scripted pipeline! Only having this feature in declarative pipeline would fall far short of the needs.

          Philip Stroh added a comment - I can just agree - from my perspective the ability to skip stages and visualize that stages are skipped is a (long awaited) must have feature for both declarative and scripted pipeline! Only having this feature in declarative pipeline would fall far short of the needs.

          I find unfortunate that the ticket be rejected wholesale.

          I think it had 2 things potentially in its scope that could be considered independently, and both of them are pretty common sense IMHO.

          The first part is adding support for skipped stages (i.e. add an expression to the stage to determine whether to skip or not), which was shutdown, sadly.

          The second part is concerned solely on rendering a skipped stage in a pipeline visualization (classic or blue ocean). This applies to declarative pipelines right now, and I can't think there would be anyone arguing against it. IF skipped stages are one day available  on scripted pipelines, the same rendering would apply as well.

          Would there be support for splitting this ticket maybe ?

          Patrice Matignon added a comment - I find unfortunate that the ticket be rejected wholesale. I think it had 2 things potentially in its scope that could be considered independently, and both of them are pretty common sense IMHO. The first part is adding support for skipped stages (i.e. add an expression to the stage to determine whether to skip or not), which was shutdown, sadly. The second part is concerned solely on rendering a skipped stage in a pipeline visualization (classic or blue ocean). This applies to declarative pipelines right now, and I can't think there would be anyone arguing against it. IF skipped stages are one day available  on scripted pipelines, the same rendering would apply as well. Would there be support for splitting this ticket maybe ?

          I think people with scripted pipelines could benefit greatly from this. Scripted pipeline users run more advanced pipelines I'm guessing, so it feels weird to not support this feature there as it seems they will be the first ones to implement this functionality in their pipelines.

          Simon Wydooghe added a comment - I think people with scripted pipelines could benefit greatly from this. Scripted pipeline users run more advanced pipelines I'm guessing, so it feels weird to not support this feature there as it seems they will be the first ones to implement this functionality in their pipelines.

          Why is the issue marked as resolved? The declarative pipeline is a subset of scripted pipeline (if I understand correctly), so there should be also possible to skip the stage.

          Grzegorz Kotarski added a comment - Why is the issue marked as resolved? The declarative pipeline is a subset of scripted pipeline (if I understand correctly), so there should be also possible to skip the stage.

          +1
          Even when the issue proper here concerns me, I find it very worrying that there are features that are planned to be only available in declarative when I thought declarative was just a convenient simplification of scripted pipelines for people only needing very simple pipelines. Is obvious that when it comes to complex pipelines, or even sharing code (reuse possibilities are very crippled in declarative, as one can't put stages or stuff like agent or triggers in a library), there is no other option but scripted pipelines.
          It seems crazy (and backwards) to make scripted pipelines a second-class citizen.

          Leandro Lucarella added a comment - +1 Even when the issue proper here concerns me, I find it very worrying that there are features that are planned to be only available in declarative when I thought declarative was just a convenient simplification of scripted pipelines for people only needing very simple pipelines. Is obvious that when it comes to complex pipelines, or even sharing code (reuse possibilities are very crippled in declarative, as one can't put stages or stuff like agent or triggers in a library), there is no other option but scripted pipelines. It seems crazy (and backwards) to make scripted pipelines a second-class citizen.

          Ben Keil added a comment -

          Ben Keil added a comment - Take a look at  https://github.com/comquent/imperative-when

          Aron Smyk added a comment - - edited

          Works like a charm! Great find benkeil83, thanks!

          Since the solution was 15 lines of Shared Library I hope it will be implemented.

          Aron Smyk added a comment - - edited Works like a charm! Great find benkeil83 , thanks! Since the solution was 15 lines of Shared Library I hope it will be implemented.

          I can confirm too, it works very nice, good work!

          Grzegorz Kotarski added a comment - I can confirm too, it works very nice, good work!

          Reinhold Füreder added a comment - - edited

          benkeil83 Great + thanks!

          Some questions and comments WRT https://github.com/comquent/imperative-when:

          1. jamesdumay org.jenkinsci.plugins.pipeline.modeldefinition.Utils.markStageSkippedForConditional(STAGE_NAME) is annotated with @Restricted(NoExternalUse.class) => I assume one therefore should not and cannot rely on this method being available forever. However, since this approach seems to work very well or would be at least an acceptable workaround for scripted pipelines, how are the chances that this will be get part of the official API (via removing this annotation?) or might even get used/wrapped by a new official API?
            • When trying out the other similarly named methods in org.jenkinsci.plugins.pipeline.modeldefinition.Utils (i.e. markStageFailedAndContinued, markStageSkippedForFailure, markStageSkippedForUnstable, markStageSkippedForConditional) the behaviour is rather surprising and "proofs" the aforementioned annotation => so let's just stay with org.jenkinsci.plugins.pipeline.modeldefinition.Utils.markStageSkippedForConditional(STAGE_NAME)
          2. jamesdumay The mouse over hovering text in blue ocean shows "common.state.skipped": Since I also get this in a little declarative pipeline prototype I guess this is a minor bug (missing translation, or – maybe even better/preferable – missing to state the reason for the skipping) and not a side effect of (ab-)using this method in the wrong context?
          3. @ All: I am wondering why the body closure evaluation is coded with delegation based on config map in this example!?
          4. @ All: The STAGE_NAME is an unknown property in class section of shared library (that is not in "vars" section) but works in "Jenkinsfile" – I guess that is due to some transformation magic applied on "vars" section?
          5. @ All: Is that unfortunately leading to such SEVERE log lines:
            2018-04-06 11:04:03 SEVERE [io.jenkins.blueocean.rest.impl.pipeline.PipelineNodeGraphVisitor <init>]   Could not find execution for run ACME-Pipeline#2347
            • Update: No that is another Jenkins pipeline or Blue Ocean bug, because it also happens for builds that do NOT use this method/approach; or that even do not skip any stages...

          Reinhold Füreder added a comment - - edited benkeil83 Great + thanks! Some questions and comments WRT https://github.com/comquent/imperative-when: jamesdumay org.jenkinsci.plugins.pipeline.modeldefinition.Utils.markStageSkippedForConditional(STAGE_NAME) is annotated with @Restricted(NoExternalUse.class) => I assume one therefore should not and cannot rely on this method being available forever. However, since this approach seems to work very well or would be at least an acceptable workaround for scripted pipelines, how are the chances that this will be get part of the official API (via removing this annotation?) or might even get used/wrapped by a new official API? When trying out the other similarly named methods in org.jenkinsci.plugins.pipeline.modeldefinition.Utils (i.e. markStageFailedAndContinued , markStageSkippedForFailure , markStageSkippedForUnstable , markStageSkippedForConditional ) the behaviour is rather surprising and "proofs" the aforementioned annotation => so let's just stay with org.jenkinsci.plugins.pipeline.modeldefinition.Utils.markStageSkippedForConditional(STAGE_NAME) jamesdumay The mouse over hovering text in blue ocean shows "common.state.skipped": Since I also get this in a little declarative pipeline prototype I guess this is a minor bug (missing translation, or – maybe even better/preferable – missing to state the reason for the skipping) and not a side effect of (ab-)using this method in the wrong context ? @ All: I am wondering why the body closure evaluation is coded with delegation based on config map in this example!? @ All: The STAGE_NAME is an unknown property in class section of shared library (that is not in "vars" section) but works in "Jenkinsfile" – I guess that is due to some transformation magic applied on "vars" section? @ All: Is that unfortunately leading to such SEVERE log lines: 2018-04-06 11:04:03 SEVERE [io.jenkins.blueocean.rest.impl.pipeline.PipelineNodeGraphVisitor <init>]   Could not find execution for run ACME-Pipeline#2347 Update: No that is another Jenkins pipeline or Blue Ocean bug, because it also happens for builds that do NOT use this method/approach; or that even do not skip any stages...

          Chris Maes added a comment -

          This issue is not solved. The skipping of stages in blue ocean view is well supported in Pipeline syntax, but not in scripted pipelines.

          The workaround proposed in https://github.com/comquent/imperative-when works correctly, but since I am using https://github.com/jenkinsci/JenkinsPipelineUnit it seems quite impossible for me to get my unit-tests working.

          Chris Maes added a comment - This issue is not solved. The skipping of stages in blue ocean view is well supported in Pipeline syntax, but not in scripted pipelines. The workaround proposed in https://github.com/comquent/imperative-when  works correctly, but since I am using https://github.com/jenkinsci/JenkinsPipelineUnit  it seems quite impossible for me to get my unit-tests working.

          This morning I was positively surprised that according to BlueOcean change log (https://plugins.jenkins.io/blueocean) this issue was solved:

          Change log
          1.11.0 (Feb 13, 2019)

          ...
          JENKINS-47286 Support skipping stages in scripted pipelines for nice visualization in blue ocean and classic UI stage view
          ...

          ... "only" to find out that it is still in status reopened, i.e. unresolved and that "only" one aspect was solved: namely the missing translation (https://github.com/jenkinsci/blueocean-plugin/commit/b0bf45c64afbf511971c7e79ff2dd317f1bddfc8) – of course I am happy/grateful about that too.

          However, this is a bit misleading

          Reinhold Füreder added a comment - This morning I was positively surprised that according to BlueOcean change log ( https://plugins.jenkins.io/blueocean) this issue was solved: Change log 1.11.0 (Feb 13, 2019) ... JENKINS-47286 Support skipping stages in scripted pipelines for nice visualization in blue ocean and classic UI stage view ... ... "only" to find out that it is still in status reopened, i.e. unresolved and that "only" one aspect was solved: namely the missing translation ( https://github.com/jenkinsci/blueocean-plugin/commit/b0bf45c64afbf511971c7e79ff2dd317f1bddfc8 ) – of course I am happy/grateful about that too. However, this is a bit misleading

          Cameron Taggart added a comment - - edited

          How do we use the workaround at https://github.com/comquent/imperative-when ? It does not appear to be a plugin that we can load. I'm a bit confused on how to use it. Any tips are appreciated. It would be great if this came out of the box. I'm migrating a declarative Jenkinsfile and got stuck on this.

          Cameron Taggart added a comment - - edited How do we use the workaround at https://github.com/comquent/imperative-when ? It does not appear to be a plugin that we can load. I'm a bit confused on how to use it. Any tips are appreciated. It would be great if this came out of the box. I'm migrating a declarative Jenkinsfile and got stuck on this.

          Adding it as a Global Shared Library is documented here. https://jenkins.io/doc/book/pipeline/shared-libraries/#global-shared-libraries
          It is working for me.

          Cameron Taggart added a comment - Adding it as a Global Shared Library is documented here. https://jenkins.io/doc/book/pipeline/shared-libraries/#global-shared-libraries It is working for me.

          Mihai Dinu added a comment - - edited

          There's a nice post here: **https://comquent.de/en/skipped-stages-in-jenkins-scripted-pipeline/ that describes how to use the https://github.com/comquent/imperative-when workaround. 

          I have a related question though: is it possible to show the stages that follow a failed stage as skipped (or failed)? My use case is something like this

          node {
              try {
                  stage('Will fail') {
                      sh 'exit 1'
                  }
                  stage('Some operations') {
                      echo 'This stage should run when previous stage is successful'
                      echo 'Should be skipped or failed when previous stage fails'
                  }
              } 
              catch (err) {
                  echo "Print the error from the first stage $err"
              }
              finally {
                  stage('Post build') {
                      echo 'Always do some cleanup ops'
                  }
              }
          }
          
          
          

           The problem that I'm facing is that any stage from my pipeline can fail and I want all my downstream stages to be skipped, except the post build one. When a stage fails, it is caught in the catch block and so all downstream stages are ignored, ruining the pipeline stage view from the job UI. 

          Any possible solutions for this?

          Mihai Dinu added a comment - - edited There's a nice post here: ** https://comquent.de/en/skipped-stages-in-jenkins-scripted-pipeline/  that describes how to use the https://github.com/comquent/imperative-when  workaround.  I have a related question though: is it possible to show the stages that follow a failed stage as skipped (or failed)? My use case is something like this node { try { stage( 'Will fail' ) { sh 'exit 1' } stage( 'Some operations' ) { echo 'This stage should run when previous stage is successful' echo 'Should be skipped or failed when previous stage fails' } } catch (err) { echo "Print the error from the first stage $err" } finally { stage( 'Post build' ) { echo 'Always do some cleanup ops' } } }  The problem that I'm facing is that any stage from my pipeline can fail and I want all my downstream stages to be skipped, except the post build one. When a stage fails, it is caught in the catch block and so all downstream stages are ignored, ruining the pipeline stage view from the job UI.  Any possible solutions for this?

          While I am not sure if this is the correct/perfect/recommended place for this question, it admittedly fits to this issue and I can actually give you a little hint

          My approach for this is to have a skipableStage step in my shared Jenkins pipeline library that wraps the default/standard stage step, so the usage is almost like normally:

          ...
          acmeSharedLibrary.skipableStage("stage name") {
             ... // normal stage implementation
          }
          ...
          

          ... and the steps implementation in pseudo code is:

          stage(stageName) {
            if (currentBuild.result) {
              echo "Skipping stage '${stageName}' due to build result '${currentBuild.result}'..."
              Result result = Result.fromString(currentBuild.result)
              markStageSkippedForResult(stageName, result) // via the aformentioned workaround
            } else {
              body() // which is the closure block of this "skippableStage"
            }
          }
          

          Reinhold Füreder added a comment - While I am not sure if this is the correct/perfect/recommended place for this question, it admittedly fits to this issue and I can actually give you a little hint My approach for this is to have a skipableStage step in my shared Jenkins pipeline library that wraps the default/standard stage step, so the usage is almost like normally: ... acmeSharedLibrary.skipableStage( "stage name" ) { ... // normal stage implementation } ... ... and the steps implementation in pseudo code is: stage(stageName) { if (currentBuild.result) { echo "Skipping stage '${stageName}' due to build result '${currentBuild.result}' ..." Result result = Result.fromString(currentBuild.result) markStageSkippedForResult(stageName, result) // via the aformentioned workaround } else { body() // which is the closure block of this "skippableStage" } }

          Michael Neale added a comment -

          (can't leave this unassigned for some reason). 

          Michael Neale added a comment - (can't leave this unassigned for some reason). 

          michaelneale I am a bit surprised by being the new assignee of this issue, as I am not sure what you are expecting from me. Considering the admittedly rather long history of this issue and its distractions/deviations I think I can only try to do a little update:

          Reinhold Füreder added a comment - michaelneale I am a bit surprised by being the new assignee of this issue, as I am not sure what you are expecting from me. Considering the admittedly rather long history of this issue and its distractions/deviations I think I can only try to do a little update: the issue's title/goal is still completely the same and correct and IMHO the "only" (or main) thing certainly missing is a stable/public API for org.jenkinsci.plugins.pipeline.modeldefinition.Utils.markStageSkippedForConditional(STAGE_NAME) (as this is annotated with @Restricted(NoExternalUse.class) ) In the course of doing that other questions from https://issues.jenkins-ci.org/browse/JENKINS-47286?focusedCommentId=334032&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-334032 might be answered as well... (I think item #2 "mouse hover" was actually fixed; see https://issues.jenkins-ci.org/browse/JENKINS-47286?focusedCommentId=360479&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-360479 )

          Reinhold Füreder added a comment - - edited

          Assigning this back to michaelneale with the update/summary comment above...

          Reinhold Füreder added a comment - - edited Assigning this back to michaelneale with the update/summary comment above...

          Michael Neale added a comment -

          can't seem to unassign it. 

           

          Michael Neale added a comment - can't seem to unassign it.   

            Unassigned Unassigned
            reinholdfuereder Reinhold Füreder
            Votes:
            10 Vote for this issue
            Watchers:
            42 Start watching this issue

              Created:
              Updated:
              Resolved: