Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-48980

Add ability to hide skipped stages/ignore stages altogether (support DRY)

      我已经在其他票证和网站上多次看到这一点,但我找不到专门解决它的票证。(所以希望我不只是简单地错过它)

       

      问题:给定几个不同的存储库,具有执行不同阶段的不同分支,没有简单的方法可以基于某些配置修改管道以保持声明性管道 DRY。

       

      简单的例子:

      • 分支 1 阶段:A -> B -> C
      • 分支 2 阶段:A -> B -> D
      • 分支 3 个阶段:A -> B -> C -> D

      目前,这需要定义 3 个不同的管道,这意味着冗余代码存在于不同的文件中,进而增加了发生错误的可能性。在更复杂的情况下,根本不可能查看对所有不同管道组合的更改。

      例如,在阶段中使用“when”是解决此问题的一种方法,但这会导致在分支构建中看到“部署”阶段而不是部署的开发人员感到困惑。此外,对于复杂的场景,可能会有许多跳过的阶段,这些阶段总是会被跳过,并简单地向可视化添加不必要的冗长。

      另一种解决方法是使用脚本化管道,但使用声明式提供的许多可视化功能不存在或始终无法正常工作。此外,由于阶段列表不是预先确定的,阶段视图总是在构建期间重置。

       

      我能想到的解决方案有两种:

      1. 提供修改“何时”行为的能力,让蓝海可视化完全忽略它。这种方法的问题在于,我相信在执行阶段时会评估“何时”条件,但阶段列表是在评估管道时在开始时确定的。这可能实际上可以实现,只是简单地在可视化中隐藏舞台而不是完全忽略它。例如:(这可能会大大改进,但它表明了这一点):
        config = [ 
            skipStage2 : true 
        ] 
        
        pipeline { 
            agent any 
            stage { 
                stage( "Stage 1" ) { ... } 
                stage( "Stage 2" ) { 
                    when { 
                         showOnlyWhen { 
                             expression { return !config.skipStage2 } // 在执行阶段
                         } 
                     } 
                    ... 
                } 
            } 
        }
      1.  添加另一个预先评估以确定阶段列表的指令。如果阶段列表从构建到构建发生变化,这与整个声明性管道发生变化并且存在不同阶段没有什么不同。例如:
        config = [ 
            skipStage2 : true , // 当existsStage2 = false
             existsStage2 : false 
        ]
        
        管道{ 
            agent any 
            stage { 
                stage( "Stage 1" ) { ... } 
                stage( "Stage 2" ) { 
                    when { 
                        expression { return ! config.skipStage2} //在执行阶段之后评价
                    }
                    存在{
                        表达式{返回config.existsStage2} //评估的前期确定是否 这舞台甚至运行
                    } 
                    ... 
                } 
            } 
        }

         

       

          [JENKINS-48980] Add ability to hide skipped stages/ignore stages altogether (support DRY)

          pixman20 created issue -

          Andrew Bayer added a comment -

          I think the best place to discuss/address this would be in Blue Ocean - so I'm moving this to that component. I don't think we want to change when behavior in Declarative itself, so visualization changes are the best option. Declarative already provides information on why a stage was skipped (i..e, skipped due to earlier failure, skipped due to when condition, etc) that Blue Ocean can access.

          Andrew Bayer added a comment - I think the best place to discuss/address this would be in Blue Ocean - so I'm moving this to that component. I don't think we want to change when behavior in Declarative itself, so visualization changes are the best option. Declarative already provides information on why a stage was skipped (i..e, skipped due to earlier failure, skipped due to when condition, etc) that Blue Ocean can access.
          Andrew Bayer made changes -
          Component/s New: blueocean-plugin [ 21481 ]
          Component/s Original: pipeline-model-definition-plugin [ 21706 ]
          Assignee Original: Andrew Bayer [ abayer ]

          pixman20 added a comment - - edited

          abayer, Is there already information provided by declarative so that Blue Ocean would know when to suppress showing the stage (rather than just marking it skipped)?
          Also, I was thinking that it would need to be added to declarative in order to hide the stages from the stage view on the main job page.

          pixman20 added a comment - - edited abayer , Is there already information provided by declarative so that Blue Ocean would know when to suppress showing the stage (rather than just marking it skipped)? Also, I was thinking that it would need to be added to declarative in order to hide the stages from the stage view on the main job page.
          pixman20 made changes -
          Description Original: I've seen this mentioned several times in other tickets and websites, but I was unable to find a ticket that specifically addresses it.  (So hopefully I didn't just simply miss it)

           

          Problem: Given several different repositories, with different branches that do different stages, there is no simple way to modify a pipeline based on some configuration in order to keep declarative pipelines DRY.

           

          Simple example:
           * Branch 1 stages: A -> B -> C
           * Branch 2 stages: A -> B -> D
           * Branch 3 stages: A -> B -> C -> D

          Currently, that requires defining 3 different pipelines, which means redundant code exists in different files and in turn increases the potential for errors to occur.  In more complex situations, the ability to review changes to all the different combinations of pipelines is simply not possible.

          Using "when" in stages is a workaround to this problem, but that causes confusion for developers who see a "Deploy" stage in a branch build that should not be deploying, for example.  Also, for complicated scenarios, there could be many skipped stages that will always be skipped and simply add unnecessary verbosity to the visualization.

          The other workaround is to use scripted pipelines, but many of the visualization features provided by using declarative don't exist or don't work correctly all the time.  Also, since the list of stages are not determined up-front, the stage view always resets during a build.

           

          There are 2 solutions that I can think of:
           # Provide the ability to modify the "when" behavior, so that blue ocean visualization completely ignores it.  The problem with this approach is that the "when" condition, I believe, is evaluated when executing the stage, but the list of stages is determined in the beginning at the time the pipeline is evaluated.  It's possible that this could actually be implemented though to simply hide the stage in the visualizations rather than ignore it completely.  For example: (This can probably be greatly improved upon, but it shows the point):
          {code:java}
          config = [
              skipStage2 : true
          ]

          pipeline {
              agent any
              stages {
                  stage("Stage 1") { ... }
                  stage("Stage 2") {
                      when {
                           showOnlyWhen {
                               expression { return !config.skipStage2 } // evaluated after the stage is executed
                           }
                       }
                      ...
                  }
              }
          }{code}

           #  Add another directive that is evaluated up-front to determine the stage list.  If the list of stages changes from build to build, it would be no different than if the entire declarative pipeline changed and different stages existed.  For example:
          {code:java}
          config = [
              skipStage2 : true, // irrelevant when existsStage2 = true
              existsStage2 : false
          ]

          pipeline {
              agent any
              stages {
                  stage("Stage 1") { ... }
                  stage("Stage 2") {
                      when {
                               expression { return !config.skipStage2 } // evaluated after the stage is executed
                           }
                       }
                       exists {
                           expression { return config.existsStage2 } // Evaluated up-front to determine if this stage is even run
                       }
                      ...
                  }
              }
          }{code}
           

           
          New: I've seen this mentioned several times in other tickets and websites, but I was unable to find a ticket that specifically addresses it.  (So hopefully I didn't just simply miss it)

           

          Problem: Given several different repositories, with different branches that do different stages, there is no simple way to modify a pipeline based on some configuration in order to keep declarative pipelines DRY.

           

          Simple example:
           * Branch 1 stages: A -> B -> C
           * Branch 2 stages: A -> B -> D
           * Branch 3 stages: A -> B -> C -> D

          Currently, that requires defining 3 different pipelines, which means redundant code exists in different files and in turn increases the potential for errors to occur.  In more complex situations, the ability to review changes to all the different combinations of pipelines is simply not possible.

          Using "when" in stages is a workaround to this problem, but that causes confusion for developers who see a "Deploy" stage in a branch build that should not be deploying, for example.  Also, for complicated scenarios, there could be many skipped stages that will always be skipped and simply add unnecessary verbosity to the visualization.

          The other workaround is to use scripted pipelines, but many of the visualization features provided by using declarative don't exist or don't work correctly all the time.  Also, since the list of stages are not determined up-front, the stage view always resets during a build.

           

          There are 2 solutions that I can think of:
           # Provide the ability to modify the "when" behavior, so that blue ocean visualization completely ignores it.  The problem with this approach is that the "when" condition, I believe, is evaluated when executing the stage, but the list of stages is determined in the beginning at the time the pipeline is evaluated.  It's possible that this could actually be implemented though to simply hide the stage in the visualizations rather than ignore it completely.  For example: (This can probably be greatly improved upon, but it shows the point):
          {code:java}
          config = [
              skipStage2 : true
          ]

          pipeline {
              agent any
              stages {
                  stage("Stage 1") { ... }
                  stage("Stage 2") {
                      when {
                           showOnlyWhen {
                               expression { return !config.skipStage2 } // evaluated after the stage is executed
                           }
                       }
                      ...
                  }
              }
          }{code}

           #  Add another directive that is evaluated up-front to determine the stage list.  If the list of stages changes from build to build, it would be no different than if the entire declarative pipeline changed and different stages existed.  For example:
          {code:java}
          config = [
              skipStage2 : true, // irrelevant when existsStage2 = false
              existsStage2 : false
          ]

          pipeline {
              agent any
              stages {
                  stage("Stage 1") { ... }
                  stage("Stage 2") {
                      when {
                               expression { return !config.skipStage2 } // evaluated after the stage is executed
                           }
                       }
                       exists {
                           expression { return config.existsStage2 } // Evaluated up-front to determine if this stage is even run
                       }
                      ...
                  }
              }
          }{code}
           

           
          pixman20 made changes -
          Description Original: I've seen this mentioned several times in other tickets and websites, but I was unable to find a ticket that specifically addresses it.  (So hopefully I didn't just simply miss it)

           

          Problem: Given several different repositories, with different branches that do different stages, there is no simple way to modify a pipeline based on some configuration in order to keep declarative pipelines DRY.

           

          Simple example:
           * Branch 1 stages: A -> B -> C
           * Branch 2 stages: A -> B -> D
           * Branch 3 stages: A -> B -> C -> D

          Currently, that requires defining 3 different pipelines, which means redundant code exists in different files and in turn increases the potential for errors to occur.  In more complex situations, the ability to review changes to all the different combinations of pipelines is simply not possible.

          Using "when" in stages is a workaround to this problem, but that causes confusion for developers who see a "Deploy" stage in a branch build that should not be deploying, for example.  Also, for complicated scenarios, there could be many skipped stages that will always be skipped and simply add unnecessary verbosity to the visualization.

          The other workaround is to use scripted pipelines, but many of the visualization features provided by using declarative don't exist or don't work correctly all the time.  Also, since the list of stages are not determined up-front, the stage view always resets during a build.

           

          There are 2 solutions that I can think of:
           # Provide the ability to modify the "when" behavior, so that blue ocean visualization completely ignores it.  The problem with this approach is that the "when" condition, I believe, is evaluated when executing the stage, but the list of stages is determined in the beginning at the time the pipeline is evaluated.  It's possible that this could actually be implemented though to simply hide the stage in the visualizations rather than ignore it completely.  For example: (This can probably be greatly improved upon, but it shows the point):
          {code:java}
          config = [
              skipStage2 : true
          ]

          pipeline {
              agent any
              stages {
                  stage("Stage 1") { ... }
                  stage("Stage 2") {
                      when {
                           showOnlyWhen {
                               expression { return !config.skipStage2 } // evaluated after the stage is executed
                           }
                       }
                      ...
                  }
              }
          }{code}

           #  Add another directive that is evaluated up-front to determine the stage list.  If the list of stages changes from build to build, it would be no different than if the entire declarative pipeline changed and different stages existed.  For example:
          {code:java}
          config = [
              skipStage2 : true, // irrelevant when existsStage2 = false
              existsStage2 : false
          ]

          pipeline {
              agent any
              stages {
                  stage("Stage 1") { ... }
                  stage("Stage 2") {
                      when {
                               expression { return !config.skipStage2 } // evaluated after the stage is executed
                           }
                       }
                       exists {
                           expression { return config.existsStage2 } // Evaluated up-front to determine if this stage is even run
                       }
                      ...
                  }
              }
          }{code}
           

           
          New: I've seen this mentioned several times in other tickets and websites, but I was unable to find a ticket that specifically addresses it.  (So hopefully I didn't just simply miss it)

           

          Problem: Given several different repositories, with different branches that do different stages, there is no simple way to modify a pipeline based on some configuration in order to keep declarative pipelines DRY.

           

          Simple example:
           * Branch 1 stages: A -> B -> C
           * Branch 2 stages: A -> B -> D
           * Branch 3 stages: A -> B -> C -> D

          Currently, that requires defining 3 different pipelines, which means redundant code exists in different files and in turn increases the potential for errors to occur.  In more complex situations, the ability to review changes to all the different combinations of pipelines is simply not possible.

          Using "when" in stages is a workaround to this problem, but that causes confusion for developers who see a "Deploy" stage in a branch build that should not be deploying, for example.  Also, for complicated scenarios, there could be many skipped stages that will always be skipped and simply add unnecessary verbosity to the visualization.

          The other workaround is to use scripted pipelines, but many of the visualization features provided by using declarative don't exist or don't work correctly all the time.  Also, since the list of stages are not determined up-front, the stage view always resets during a build.

           

          There are 2 solutions that I can think of:
           # Provide the ability to modify the "when" behavior, so that blue ocean visualization completely ignores it.  The problem with this approach is that the "when" condition, I believe, is evaluated when executing the stage, but the list of stages is determined in the beginning at the time the pipeline is evaluated.  It's possible that this could actually be implemented though to simply hide the stage in the visualizations rather than ignore it completely.  For example: (This can probably be greatly improved upon, but it shows the point):
          {code:java}
          config = [
              skipStage2 : true
          ]

          pipeline {
              agent any
              stages {
                  stage("Stage 1") { ... }
                  stage("Stage 2") {
                      when {
                           showOnlyWhen {
                               expression { return !config.skipStage2 } // evaluated after the stage is executed
                           }
                       }
                      ...
                  }
              }
          }{code}

           #  Add another directive that is evaluated up-front to determine the stage list.  If the list of stages changes from build to build, it would be no different than if the entire declarative pipeline changed and different stages existed.  For example:
          {code:java}
          config = [
              skipStage2 : true, // irrelevant when existsStage2 = false
              existsStage2 : false
          ]

          pipeline {
              agent any
              stages {
                  stage("Stage 1") { ... }
                  stage("Stage 2") {
                      when {
                          expression { return !config.skipStage2 } // evaluated after the stage is executed
                      }
                      exists {
                          expression { return config.existsStage2 } // Evaluated up-front to determine if this stage is even run
                      }
                      ...
                  }
              }
          }{code}
           

           

          Michael Neale added a comment -

          The problem is - before skipped were show, people would complain about not showing them (assuming going with the single pipeline). Adding more config/complexity for this case.. maybe the multiple pipeline is the way if the developers really don't want to know about deploy? not keen on this as it is at all based on past history. 

          Michael Neale added a comment - The problem is - before skipped were show, people would complain about not showing them (assuming going with the single pipeline). Adding more config/complexity for this case.. maybe the multiple pipeline is the way if the developers really don't want to know about deploy? not keen on this as it is at all based on past history. 

          Timur Batyrshin added a comment - - edited

          abayer, this could be done as an additional modificator to the pipeline, something like this:

          pipeline {
            stages {
              stage('hide me') {
                when { expression { false } }
                hide true
                steps { ... }
              }
              stage('skip me but show me') {
                when { expression { false } }
          //    hide false // default
                steps { ... }
              }
              stage('run me and show me') {
                when { expression { true } }
                hide true // hiding doesn't make sense when stage is not skipped
                steps { ... }
              }
            }
          }
          
          

          My use case is a pipeline library consisting of dozen steps which is shared across dozens different projects.

          Some steps are skipped based on branch name and showing them up is ok.

          Some steps are skipped based on project type and I would want to hide them.

           

          I could keep a copy of the pipeline for every project type but that doesn't look like very much viable option as the pipeline is 700+ LOC and I don't see much options of reducing it below 400-500 LOC which is quite much too.

          Copying becomes error prone and increases maintenance when doing changes to the pipeline stages common for all pipelines.

          So I see 2 viable options for improvement:

          Any thoughts?

           

           

          Timur Batyrshin added a comment - - edited abayer , this could be done as an additional modificator to the pipeline, something like this: pipeline { stages { stage( 'hide me' ) { when { expression { false } } hide true steps { ... } } stage( 'skip me but show me' ) { when { expression { false } } // hide false // default steps { ... } } stage( 'run me and show me' ) { when { expression { true } } hide true // hiding doesn't make sense when stage is not skipped steps { ... } } } } My use case is a pipeline library consisting of dozen steps which is shared across dozens different projects. Some steps are skipped based on branch name and showing them up is ok. Some steps are skipped based on project type and I would want to hide them.   I could keep a copy of the pipeline for every project type but that doesn't look like very much viable option as the pipeline is 700+ LOC and I don't see much options of reducing it below 400-500 LOC which is quite much too. Copying becomes error prone and increases maintenance when doing changes to the pipeline stages common for all pipelines. So I see 2 viable options for improvement: add the ability to hide some changes add the ability to offload stages to external files ( https://issues.jenkins-ci.org/browse/JENKINS-42224 ) Any thoughts?    

          I would also find it very useful. I like the syntax proposed by erthad.

          My use case is the same. A default pipeline for many projects. They build the code the same way, but some projects are only pushing artifacts, other projects are building docker image.

          Romain Marteau added a comment - I would also find it very useful. I like the syntax proposed by erthad . My use case is the same. A default pipeline for many projects. They build the code the same way, but some projects are only pushing artifacts, other projects are building docker image.
          Prashanth H made changes -
          Labels New: stageview-frontend

            olamy Olivier Lamy
            pixman20 pixman20
            Votes:
            26 Vote for this issue
            Watchers:
            27 Start watching this issue

              Created:
              Updated: