Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-44094

Hide specific steps in build view

    XMLWordPrintable

    Details

    • Similar Issues:
    • Sprint:
      Blue Ocean 1.1-beta4, Blue Ocean 1.1

      Description

      We have a company-wide Pipeline setup that uses global pipeline scripts. These scripts do all sort of "behind the scenes" things to make the lives of developers as easy as possible, and make going from first commit to a testable application on Jenkins cost the least amount of code/configuration.

      This is all working great, and developers don't really interact with Jenkins in any way, unless their build failed, and they want to know why.

      Right now, when they click the red cross on GitHub, they are redirected to the build status on the "blue" view, but then they are greeted with multiple steps, that have no meaning to them, and are (given our global scripts work as expected) not the cause of their problems, they just want to see the output of their test run, not all the surrounding steps that make it possible to run their build/test.

      So basically, what I'm looking for is two things:

      1. make step titles more descriptive (but this is already being tackled in another issue)
      2. add the option to hide specific steps from showing up, so devs only see what's relevant

      I did find this issue: https://issues.jenkins-ci.org/browse/JENKINS-26987

      But this is not about the blueocean plugin, and I do believe that the notion "Running Steps is itself intended to be a debugging view" does not apply here, as the blueocean view very much is intended to be as simple as possible for developers to get to the root of their problems, and not for the "Jenkins people" amongst us, who want to drill down into all the possible details there are (for which we still have the classic interface).

        Attachments

          Issue Links

            Activity

            jeanmertz Jean Mertz created issue -
            Hide
            jamesdumay James Dumay added a comment -

            Jean Mertz alternatively have you thought about wrapping these steps in a stage to differentiate them from the developers portion of the pipeline?

            We are hoping to release JENKINS-37324 in Blue Ocean 1.1 shortly which should improve the step descriptions.

            Show
            jamesdumay James Dumay added a comment - Jean Mertz alternatively have you thought about wrapping these steps in a stage to differentiate them from the developers portion of the pipeline? We are hoping to release JENKINS-37324 in Blue Ocean 1.1 shortly which should improve the step descriptions.
            Hide
            jeanmertz Jean Mertz added a comment -

            Thanks for the pointer about the step description, James Dumay.

            Unfortunately, using different stages is not really the solution we're looking for, since we have a lot of automated steps before/after different stages within our pipeline. So we'd get stages like:

            • pre-setup
            • setup
            • post-setup
            • pre-test
            • test
            • post-test
            • pre-deploy
            • deploy
            • post-deploy

            And then we'd have to inform all devs "don't concern yourself with the pre and post stages, only look at the setup, test and deploy stages".

            This doesn't make for a great user experience.

            Show
            jeanmertz Jean Mertz added a comment - Thanks for the pointer about the step description, James Dumay . Unfortunately, using different stages is not really the solution we're looking for, since we have a lot of automated steps before/after different stages within our pipeline. So we'd get stages like: pre-setup setup post-setup pre-test test post-test pre-deploy deploy post-deploy And then we'd have to inform all devs "don't concern yourself with the pre and post stages, only look at the setup, test and deploy stages". This doesn't make for a great user experience.
            Hide
            jamesdumay James Dumay added a comment - - edited

            Jean Mertz lets explore this a bit more. What happens if one of your automated steps fails? Should we show that step? Should we show all the automated steps? When the automated steps are executing what should the user see?

            Show
            jamesdumay James Dumay added a comment - - edited Jean Mertz lets explore this a bit more. What happens if one of your automated steps fails? Should we show that step? Should we show all the automated steps? When the automated steps are executing what should the user see?
            Hide
            jeanmertz Jean Mertz added a comment -

            Good questions James Dumay.

            Currently, even for steps outside of a stage (at least at the start of a run), you see them, until a step within a stage starts, at which point the other steps go away and you only see the steps within a stage.

            I think hiding a step should be an "advanced" feature which comes with some warnings, one of them being: only hide steps which you know won't fail, or else the "simple" view in the blue job overview will show the job as failed, but not why it failed (you can always use the "escape" button to go to the classic view and see the entire log output).

            That would be the most simplistic MVP version as far as I'm concerned.

            Obviously, showing the failed step when it fails, even if it was hidden at first, would be the preferred solution, as it  will not concern the devs 99.9% of the time when that step succeeds, but if things do go wrong, it can be shown, and they can ask anyone who is knowledgable about the shared-pipeline steps what is going on.

            As far as "what to show while executing" is concerned, I think the same applies as before: I think it makes sense to collapse the step before, but not show/expand the next step yet, until any steps in-between that are hidden are completed.

            This is obviously a trade-off, as hiding anything can mask problems/hangs/etc, but that's I believe the whole point of this feature: as a company, you can have steps that you mark as "irrelevant to the devs" and also "99.9% safe", after which you can hide that step, and improve readability/debugging in most use-cases, while still having a backup (the default Jenkins logs) in case things do go wrong.

            Does that answer your questions?

            Show
            jeanmertz Jean Mertz added a comment - Good questions James Dumay . Currently, even for steps outside of a stage (at least at the start of a run), you see them, until a step within a stage starts, at which point the other steps go away and you only see the steps within a stage. I think hiding a step should be an "advanced" feature which comes with some warnings, one of them being: only hide steps which you know won't fail, or else the "simple" view in the blue job overview will show the job as failed, but not why it failed (you can always use the "escape" button to go to the classic view and see the entire log output). That would be the most simplistic MVP version as far as I'm concerned. Obviously, showing the failed step when it fails, even if it was hidden at first, would be the preferred solution, as it  will not concern the devs 99.9% of the time when that step succeeds, but if things do go wrong, it can be shown, and they can ask anyone who is knowledgable about the shared-pipeline steps what is going on. As far as "what to show while executing" is concerned, I think the same applies as before: I think it makes sense to collapse the step before, but not show/expand the next step yet, until any steps in-between that are hidden are completed. This is obviously a trade-off, as hiding anything can mask problems/hangs/etc, but that's I believe the whole point of this feature: as a company, you can have steps that you mark as "irrelevant to the devs" and also "99.9% safe", after which you can hide that step, and improve readability/debugging in most use-cases, while still having a backup (the default Jenkins logs) in case things do go wrong. Does that answer your questions?
            Hide
            jamesdumay James Dumay added a comment - - edited

            Jean Mertz I am very hesitant of showing and hiding steps based on the authors desired visibility level or state - we should show steps consistently or not at all - however, we could consider something like:

            steps {
               echo 'A visible step'
               group ('Common steps') {
                  echo 'shown only within common steps'
                  echo 'another step'
               }
               echo 'Another visible step'
            }
            

            So that the user sees:

            • A visible step
            • Common steps
            • Another visible step

            ... instead of the full step listing.

            Andrew Bayer Michael Neale if we had some sanctioned way to group N steps to show as one step, this could also simplify the cases where we have steps that are made of other steps and we get "failing steps" that we don't care about (because we catch them and do something else).

            I believe the docker Pipeline code has something like:

               try {
                   docker run $image
               } catch (Exception e) {
                   docker pull $image
                   docker run $image
               }
            

            The step within the try will show as failed if we don't have the image then have two successful steps after it. We'd want to visualise that block as one "group" so this syntax could be used like so:

            group('docker stuff') {
              try {
                   docker run $image
               } catch (Exception e) {
                   docker pull $image
                   docker run $image
               }
            }
            

            If the steps within the try fail and the rest succeed we would visualise this as one step "docker stuff" that succeeded.

            Show
            jamesdumay James Dumay added a comment - - edited Jean Mertz I am very hesitant of showing and hiding steps based on the authors desired visibility level or state - we should show steps consistently or not at all - however, we could consider something like: steps { echo 'A visible step' group ( 'Common steps' ) { echo 'shown only within common steps' echo 'another step' } echo 'Another visible step' } So that the user sees: A visible step Common steps Another visible step ... instead of the full step listing. Andrew Bayer Michael Neale if we had some sanctioned way to group N steps to show as one step, this could also simplify the cases where we have steps that are made of other steps and we get "failing steps" that we don't care about (because we catch them and do something else). I believe the docker Pipeline code has something like: try { docker run $image } catch (Exception e) { docker pull $image docker run $image } The step within the try will show as failed if we don't have the image then have two successful steps after it. We'd want to visualise that block as one "group" so this syntax could be used like so: group( 'docker stuff' ) { try { docker run $image } catch (Exception e) { docker pull $image docker run $image } } If the steps within the try fail and the rest succeed we would visualise this as one step "docker stuff" that succeeded.
            jamesdumay James Dumay made changes -
            Field Original Value New Value
            Assignee James Dumay [ jamesdumay ]
            Hide
            jamesdumay James Dumay added a comment -

            Assigning to myself while I drive out requirements.

            Show
            jamesdumay James Dumay added a comment - Assigning to myself while I drive out requirements.
            jamesdumay James Dumay made changes -
            Sprint Blue Ocean 1.1-beta3 [ 316 ]
            jamesdumay James Dumay made changes -
            Rank Ranked lower
            jamesdumay James Dumay made changes -
            Sprint Blue Ocean 1.1-beta4 [ 316 ] Blue Ocean 1.1-beta4, Blue Ocean 1.1-beta5 [ 316, 321 ]
            Hide
            jamesdumay James Dumay added a comment -

            Jean Mertz what do you think of the proposal I wrote above?

            Show
            jamesdumay James Dumay added a comment - Jean Mertz what do you think of the proposal I wrote above?
            Hide
            jeanmertz Jean Mertz added a comment -

            I think grouping of steps would certainly be a great addition and improve the situation by a lot.

            Regarding this quote:

            > I am very hesitant of showing and hiding steps based on the authors desired visibility level or state

            What I am proposing is that step visibility be defined in a Jenkinsfile (or in our case, the global-pipeline-script code that is executed for each Jenkinsfile in our organisation). This way, it's a conscious decision made by the people maintaining the Jenkinsfile of a project or organisation.

            Another thing I noticed is that currently any steps that run after the last stage in a pipeline is completed, are not shown in the interface, and in fact it is possible to have a failed run (with a red bar at the top), without any failed stages (all stages are green, no steps are red).

            Show
            jeanmertz Jean Mertz added a comment - I think grouping of steps would certainly be a great addition and improve the situation by a lot. Regarding this quote: > I am very hesitant of showing and hiding steps based on the authors desired visibility level or state What I am proposing is that step visibility be defined in a Jenkinsfile (or in our case, the global-pipeline-script code that is executed for each Jenkinsfile in our organisation). This way, it's a conscious decision made by the people maintaining the Jenkinsfile of a project or organisation. Another thing I noticed is that currently any steps that run  after the last stage in a pipeline is completed, are not shown in the interface, and in fact it is possible to have a failed run (with a red bar at the top), without any failed stages (all stages are green, no steps are red).
            Hide
            jamesdumay James Dumay added a comment -

            Another thing I noticed is that currently any steps that run after the last stage in a pipeline is completed, are not shown in the interface, and in fact it is possible to have a failed run (with a red bar at the top), without any failed stages (all stages are green, no steps are red).

            I assume you've got a Pipeline that looks like

            node {
               stage('stuff') {
                 echo 'can be seen in ui'
               }
               echo('cannot be seen in ui');
            }
            

            The workaround here is to wrap those steps within a stage. We determined a while ago that we cannot reliably attach orphaned steps to the nearest stage or parallel and have it be what most people want.

            Show
            jamesdumay James Dumay added a comment - Another thing I noticed is that currently any steps that run after the last stage in a pipeline is completed, are not shown in the interface, and in fact it is possible to have a failed run (with a red bar at the top), without any failed stages (all stages are green, no steps are red). I assume you've got a Pipeline that looks like node { stage( 'stuff' ) { echo 'can be seen in ui' } echo( 'cannot be seen in ui' ); } The workaround here is to wrap those steps within a stage. We determined a while ago that we cannot reliably attach orphaned steps to the nearest stage or parallel and have it be what most people want.
            Hide
            jamesdumay James Dumay added a comment -

            Jean Mertz as a general rule we do not want to hide steps for that exact reason. There are design problems in the original Pipeline script syntax that allowed a developer to write Pipelines that can't be visualised such as the orphaned step issue above and nesting stages/parallels.

            If we hide your global library steps from the visualization then we run into the same problem (all steps are green but the Pipeline shows failed) and that is a bigger usability issue than the one we are trying to solve here (hiding steps).

            We won't be providing a way to hide the steps. However, we can consider grouping steps into a single step so at least there are a smaller amount of steps visualized.

            Show
            jamesdumay James Dumay added a comment - Jean Mertz as a general rule we do not want to hide steps for that exact reason. There are design problems in the original Pipeline script syntax that allowed a developer to write Pipelines that can't be visualised such as the orphaned step issue above and nesting stages/parallels. If we hide your global library steps from the visualization then we run into the same problem (all steps are green but the Pipeline shows failed) and that is a bigger usability issue than the one we are trying to solve here (hiding steps). We won't be providing a way to hide the steps. However, we can consider grouping steps into a single step so at least there are a smaller amount of steps visualized.
            jamesdumay James Dumay made changes -
            Sprint Blue Ocean 1.1-beta4, Blue Ocean 1.1 [ 316, 321 ] Blue Ocean 1.1-beta4, Blue Ocean 1.1, Blue Ocean 1.1 [ 316, 321, 331 ]
            jamesdumay James Dumay made changes -
            Epic Link JENKINS-43952 [ 181484 ]
            jamesdumay James Dumay made changes -
            Sprint Blue Ocean 1.1-beta4, Blue Ocean 1.1, Blue Ocean 1.1 [ 316, 321, 331 ] Blue Ocean 1.4, Blue Ocean 1.1-beta4, Blue Ocean 1.1 [ 311, 316, 321 ]
            jamesdumay James Dumay made changes -
            Sprint Blue Ocean 1.4, Blue Ocean 1.1-beta4, Blue Ocean 1.1 [ 311, 316, 321 ] Blue Ocean 1.1-beta4, Blue Ocean 1.1 [ 316, 321 ]
            jamesdumay James Dumay made changes -
            Rank Ranked higher
            Hide
            vchijwani Vicky Chijwani added a comment - - edited

            This is a huge problem for us too. We have 100s of developers relying on a global shared library that abstracts out a lot of common functionality, exactly as described in the original issue description. It would be great to see the group proposal above implemented - it would definitely solve most of the problem.

            Show
            vchijwani Vicky Chijwani added a comment - - edited This is a huge problem for us too. We have 100s of developers relying on a global shared library that abstracts out a lot of common functionality, exactly as described in the original issue description. It would be great to see the  group proposal above implemented - it would definitely solve most of the problem.
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-37324 [ JENKINS-37324 ]
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-26987 [ JENKINS-26987 ]
            Hide
            anton_yurchenko Anton Yurchenko added a comment -

            I have an example of situations when hiding stages are required.

            Multi-branch pipeline, when push on different branch should create a different slave node in different network segment.

            In order to filter specific use case we are using WHEN condition.

            In such case, there are lots of 'pseudo' stages like "Branch: dev/master", "Branch: dev", "Env: development", "Env: production", "Branch: master".

            Such drill down also breaks the console output as two stages running in UI (for example: Branch: dev and Spawn Slave)

            stage("Branch: dev/master"){
              parallel {
                stage("Env: development") {
                  when {
                    allOf {
                      branch 'dev'
                      environment name: 'CHANGE_TARGET', value: ''
                    }
                  }
                  stages {
                    stage("Branch: dev"){
                      agent {label "development-slave"}
                      stages {
                        stage("Spawn Slave") {
                          steps {
                            sh 'command'
                          }
                          post {
                            failure {
                              sh 'command'
                            }
                            aborted {
                              sh 'command'
                            }
                          }
                        }
                        stage("Base Image") {
                          steps {
                            sh 'command'
                          }
                          post {
                            aborted {
                              sh 'command'
                            }
                            failure {
                              sh 'command'
                            }
                          }
                        }
                        stage("Dev Image") {
                          steps {
                            sh 'command'
                          }
                          post {
                            aborted {
                              sh 'command'
                            }
                            failure {
                              sh 'command'
                            }
                          }
                        }
                        stage("Notify Team"){
                          agent {label "master"}
                          steps {
                            sh 'command'
                          }
                        }
                        stage('Deploy QA') {
                          steps {
                            sh 'command'
                          }
                          post {
                            success {
                              sh 'command'
                            }
                            failure {
                              sh 'command'
                            }
                            aborted {
                              sh 'command'
                            }
                            always {
                              sh 'command'
                            }
                          }
                        }
                      }
                    }
                  }
                }
                stage("Env: production") {
                  when {
                    allOf {
                      branch 'master'
                      environment name: 'CHANGE_TARGET', value: ''
                    }
                  }
                  stages {
                    stage("Branch: master"){
                      agent {label "production-slave"}
                      stages {
                        stage("Spawn Slave") {
                          steps {
                            sh 'command'
                          }
                          post {
                            failure {
                              sh 'command'
                            }
                            aborted {
                              sh 'command'
                            }
                          }
                        }
                        stage("Base Image") {
                          steps {
                            sh 'command'
                          }
                          post {
                            aborted {
                              sh 'command'
                            }
                            failure {
                              sh 'command'
                            }
                          }
                        }
                        stage("Prod Image") {
                          steps {
                            sh 'command'
                          }
                          post {
                            success {
                              sh ' command'
                            }
                            failure {
                              sh 'command'
                            }
                            aborted {
                              sh 'command'
                            }
                            always {
                              sh 'command'
                            }
                          }
                        }
                      }
                    }
                  }
                }
              }
            }
            
            Show
            anton_yurchenko Anton Yurchenko added a comment - I have an example of situations when hiding stages are required. Multi-branch pipeline, when push on different branch should create a different slave node in different network segment. In order to filter specific use case we are using WHEN condition. In such case, there are lots of 'pseudo' stages like "Branch: dev/master", "Branch: dev", "Env: development", "Env: production", "Branch: master". Such drill down also breaks the console output as two stages running in UI (for example: Branch: dev and Spawn Slave) stage( "Branch: dev/master" ){ parallel { stage( "Env: development" ) { when { allOf { branch 'dev' environment name: 'CHANGE_TARGET' , value: '' } } stages { stage( "Branch: dev" ){ agent {label "development-slave" } stages { stage( "Spawn Slave" ) { steps { sh 'command' } post { failure { sh 'command' } aborted { sh 'command' } } } stage( "Base Image" ) { steps { sh 'command' } post { aborted { sh 'command' } failure { sh 'command' } } } stage( "Dev Image" ) { steps { sh 'command' } post { aborted { sh 'command' } failure { sh 'command' } } } stage( "Notify Team" ){ agent {label "master" } steps { sh 'command' } } stage( 'Deploy QA' ) { steps { sh 'command' } post { success { sh 'command' } failure { sh 'command' } aborted { sh 'command' } always { sh 'command' } } } } } } } stage( "Env: production" ) { when { allOf { branch 'master' environment name: 'CHANGE_TARGET' , value: '' } } stages { stage( "Branch: master" ){ agent {label "production-slave" } stages { stage( "Spawn Slave" ) { steps { sh 'command' } post { failure { sh 'command' } aborted { sh 'command' } } } stage( "Base Image" ) { steps { sh 'command' } post { aborted { sh 'command' } failure { sh 'command' } } } stage( "Prod Image" ) { steps { sh 'command' } post { success { sh ' command' } failure { sh 'command' } aborted { sh 'command' } always { sh 'command' } } } } } } } } }
            Hide
            brunomiguel_teixeira bruno teixeira added a comment - - edited

            I also have a specific use case in my company. We use jenkins & global shared libraries extensively to setup the build's workspaces.

            There are several steps (like creating folders, setting packages feeds, injecting env variables...) that i would like to "hide" from the users so that the commands that the users actually run are not mixed with shared libraries. This is because the steps in the shared libraries are not the end user's concern and often it becomes a bundled mess from the user's perspective.

             

            This is not a hard requirement, merely a UX improvement that i would like.

            Show
            brunomiguel_teixeira bruno teixeira added a comment - - edited I also have a specific use case in my company. We use jenkins & global shared libraries extensively to setup the build's workspaces. There are several steps (like creating folders, setting packages feeds, injecting env variables...) that i would like to "hide" from the users so that the commands that the users actually run are not mixed with shared libraries. This is because the steps in the shared libraries are not the end user's concern and often it becomes a bundled mess from the user's perspective.   This is not a hard requirement, merely a UX improvement that i would like.
            Hide
            gmarquez GMarquez added a comment -

            I think the ability to hide stages is useful. There are several small trivial stages in my pipeline and would like the ability to hide if possible. It would be nice to not see in OceanBlue but perhaps could still log in standard console output (in case any issues). 

            Topic: Add ability to hide skipped stages/ignore stages altogether (support DRY) , similar but not same, I like Timur Batyrshin's suggestion:
            https://issues.jenkins-ci.org/browse/JENKINS-48980?focusedCommentId=345319&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-345319

            hide true 
            // hide false // default

            Show
            gmarquez GMarquez added a comment - I think the ability to hide stages is useful. There are several small trivial stages in my pipeline and would like the ability to hide if possible. It would be nice to not see in OceanBlue but perhaps could still log in standard console output (in case any issues).  Topic: Add ability to hide skipped stages/ignore stages altogether (support DRY) , similar but not same, I like Timur Batyrshin's suggestion: https://issues.jenkins-ci.org/browse/JENKINS-48980?focusedCommentId=345319&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-345319 hide true  // hide false // default
            thomaspeitz Thomas Peitz made changes -
            Hide
            thomaspeitz Thomas Peitz added a comment - - edited

            We are using a shared jenkins library for our company which allows our developers to set up pipelines pretty easily.

            In the configuration of the pipelines we allow to enable / disable steps. Here is an example of a Jenkinsfile:

             

            @Library('ivx@0.1.0') _

             

            buildPipelineKubernetes {

               productionDeployment = false

               lintOpenApi = false

            }

             

            It works pretty good for us and removed the need that developer need to know how Jenkins works. It boosted developer happiness enormously.

             

            Now our deployment pipelines have a lot of steps which are just "white" as they are not executed for the project. For this particular project steps won't be ever executed. So it bloats the view and gives devs bad user experience.

             

            So for us it would be nice to be able to hide steps if they are not executed. Is there any chance we can implement this feature ourselves? - We would take time and try to create a pull request if there is a solution which would be accepted.

             

            We would favor the contribution rather of going back to scripted pipelines again.

             

            Greetings, Thomas 

            Show
            thomaspeitz Thomas Peitz added a comment - - edited We are using a shared jenkins library for our company which allows our developers to set up pipelines pretty easily. In the configuration of the pipelines we allow to enable / disable steps. Here is an example of a Jenkinsfile:   @Library('ivx@0.1.0') _   buildPipelineKubernetes {    productionDeployment = false    lintOpenApi = false }   It works pretty good for us and removed the need that developer need to know how Jenkins works. It boosted developer happiness enormously.   Now our deployment pipelines have a lot of steps which are just "white" as they are not executed for the project. For this particular project steps won't be ever executed. So it bloats the view and gives devs bad user experience.   So for us it would be nice to be able to hide steps if they are not executed. Is there any chance we can implement this feature ourselves? - We would take time and try to create a pull request if there is a solution which would be accepted.   We would favor the contribution rather of going back to scripted pipelines again.   Greetings, Thomas 
            framillien Florian Ramillien made changes -
            Link This issue is related to JENKINS-61195 [ JENKINS-61195 ]

              People

              Assignee:
              jamesdumay James Dumay
              Reporter:
              jeanmertz Jean Mertz
              Votes:
              20 Vote for this issue
              Watchers:
              18 Start watching this issue

                Dates

                Created:
                Updated: