Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-39628

Handle skipped stages and parallels in the Pipeline Graph

    • Blue Ocean 1.2-beta3, Blue Ocean 1.2-beta4

      Ready for release

      This improvement is completed and scheduled for release in 1.2. Check the roadmap page for updates.

      Context
      The Pipeline team have added the ability to "skip" stages in the Pipeline (JENKINS-37781).

      Scope

      • Skipped stages are shown as not built
      • Declarative Pipeline will not provide flow nodes for skipped parallels so we can always assume that there won't be any skipped parallels.
      • If the first stage is skipped then the pipeline should start from a "Start" node.
      • If the last stage is skipped then the pipeline should end with a "End" node.
      • Ensure that the editors start/end nodes match the style in this mockup

      Mockup

          [JENKINS-39628] Handle skipped stages and parallels in the Pipeline Graph

          James Dumay added a comment -

          brody we've got to come up with a way of demonstrating this.

          James Dumay added a comment - brody we've got to come up with a way of demonstrating this.

          Brody Maclean added a comment -

          jamesdumay – Initial idea

          Brody Maclean added a comment - jamesdumay – Initial idea

          Brody Maclean added a comment - - edited

          Zeplin https://zpl.io/Z2m0Q0l

          Start / End nodes are added when First / Last stages are skipped. (see bottom of mockup)
          Example of skipping 2 stages also at the bottom of the mockup

          Brody Maclean added a comment - - edited Zeplin https://zpl.io/Z2m0Q0l Start / End nodes are added when First / Last stages are skipped. (see bottom of mockup) Example of skipping 2 stages also at the bottom of the mockup

          Josh McDonald added a comment -

          Couple of questions:

          1. Is there a compelling reason to only show start/end nodes when the first/last is skipped?
          2. Do we want to visually discriminate between "skipped because of new skip method" and "not run because previous stage failed"?
          3. Related to #2, let's say I have 10 parallel stages, but my infrastructure can only schedule 3 at a time, and one of the first 3 fails. Does Jenkins "skip" the other 7? If so, how do we want them to look?

          Josh McDonald added a comment - Couple of questions: Is there a compelling reason to only show start/end nodes when the first/last is skipped? Do we want to visually discriminate between "skipped because of new skip method" and "not run because previous stage failed"? Related to #2, let's say I have 10 parallel stages, but my infrastructure can only schedule 3 at a time, and one of the first 3 fails. Does Jenkins "skip" the other 7? If so, how do we want them to look?

          Brody Maclean added a comment -

          sophistifunk
          1. I'm happy to them shown consistently, as long as we have a fallback if width starts becoming an issue.
          2. IMO a stage that hasn't run, is a stage that hasn't run, regardless of it being skipped or not run because of previous failure.
          3. If they're in "parallel" i'd run them all even if an earlier one fails, as the user would expect them all to be run

          Brody Maclean added a comment - sophistifunk 1. I'm happy to them shown consistently, as long as we have a fallback if width starts becoming an issue. 2. IMO a stage that hasn't run, is a stage that hasn't run, regardless of it being skipped or not run because of previous failure. 3. If they're in "parallel" i'd run them all even if an earlier one fails, as the user would expect them all to be run

          Michael Neale added a comment -

          Bumped this up a bit - as a few people are asking for this as it is a most useful feature when they migrate to pipeline. This looks like a non trivial bit of work, so perhaps could be split so that a first pass just shows the node correctly as some skipped visual state before the svg code is changed - thoughts jamesdumay

          Michael Neale added a comment - Bumped this up a bit - as a few people are asking for this as it is a most useful feature when they migrate to pipeline. This looks like a non trivial bit of work, so perhaps could be split so that a first pass just shows the node correctly as some skipped visual state before the svg code is changed - thoughts jamesdumay

          James Dumay added a comment -

          michaelneale this is on the plan for the pipeline papercuts post-editor work.

          James Dumay added a comment - michaelneale this is on the plan for the pipeline papercuts post-editor work.

          Since I also just stumbled over the linked issue JENKINS-44464 reported by wimnat – (a) confusing/surprising colour and (b) "Unknow" typo in title, where ideally even the reason should be stated – maybe a more pragmatic (first?) approach could be:

          • replace purple with white colour (i.e. just a black circle with no/white filling)
          • replace title message "Unknow" with "Skipped due to [expression|branch|...]" (to give a reason, but presumably avoiding giving too many details)
          • leave the stage connecting lines as is

          Reinhold Füreder added a comment - Since I also just stumbled over the linked issue JENKINS-44464 reported by wimnat – (a) confusing/surprising colour and (b) "Unknow" typo in title, where ideally even the reason should be stated – maybe a more pragmatic (first?) approach could be: replace purple with white colour (i.e. just a black circle with no/white filling) replace title message "Unknow" with "Skipped due to [expression|branch|...] " (to give a reason, but presumably avoiding giving too many details) leave the stage connecting lines as is

          Josh McDonald added a comment -

          Purple isn't "skipped", it's "unknown status" - it's meant to stand out because it means either the UI needs to be updated or the server is sending bad data.

          Josh McDonald added a comment - Purple isn't "skipped", it's "unknown status" - it's meant to stand out because it means either the UI needs to be updated or the server is sending bad data.

          Rob White added a comment -

           I would like to be able to completely hide skipped stages.

          I have a pipeline that does a different build depending on branch - i don't really want to see something like this:

          Checkout -> Build (develop) (SKIPPED) -> Build (master) -> Build (feature) (SKIPPED) -> Deploy

          It would be great if there was a per project toggle that would hide the skipped steps.  With this toggle on it would look something like:

          Checkout -> Build (master) -> Deploy

          Rob White added a comment -  I would like to be able to completely hide skipped stages. I have a pipeline that does a different build depending on branch - i don't really want to see something like this: Checkout -> Build (develop) (SKIPPED) -> Build (master) -> Build (feature) (SKIPPED) -> Deploy It would be great if there was a per project toggle that would hide the skipped steps.  With this toggle on it would look something like: Checkout -> Build (master) -> Deploy

          Thuy Do added a comment - - edited

          I second Mr. White.  In my work, I have a pipeline that contains more than 20 stages.  Some can be executed in parallel and some have to be sequential.  Each stage is executed based on a condition.  I would like to hide the stage that is not executed.  See below screenshot. 

           

          For this pipeline, I have to scroll to the right often to see the status of the stage.  It would be nice if the skipped stages either collapsed into "skipped stages" to not display at all.  I prefer not to display them at all.  Collapsing the skipped stages may not work if every other stage is skipped.

           

           

          Thuy Do added a comment - - edited I second Mr. White.  In my work, I have a pipeline that contains more than 20 stages.  Some can be executed in parallel and some have to be sequential.  Each stage is executed based on a condition.  I would like to hide the stage that is not executed.  See below screenshot.    For this pipeline, I have to scroll to the right often to see the status of the stage.  It would be nice if the skipped stages either collapsed into "skipped stages" to not display at all.  I prefer not to display them at all.  Collapsing the skipped stages may not work if every other stage is skipped.    

          Michael Neale added a comment -

          thuybdojenkins wimnat could I see a sanitised sample of your pipelines if possible? 

          Michael Neale added a comment - thuybdojenkins wimnat could I see a sanitised sample of your pipelines if possible? 

          James Dumay added a comment - - edited

          thuybdojenkins could you please open up a new feature request for skipping with an example Pipeline? Reiterating the use case you have here would be good (generally we do not accept feature requests on already resolved tickets for other features)

          James Dumay added a comment - - edited thuybdojenkins could you please open up a new feature request for skipping with an example Pipeline? Reiterating the use case you have here would be good (generally we do not accept feature requests on already resolved tickets for other features)

          Thuy Do added a comment -

          michaelneale I attached the pipeline view in my comment.  Do you need the groovy script I use?  If so, I will include in the new feature request as suggested by jamesdumay.

          Here is a snippet of a stage in my pipeline.  Other stages are similar.

          stage("dps") {

              when {
                  expression { return params.enableDps }
              }

              steps {
                  timeout(time: 10, unit: 'HOURS') {
                      ecdmItestRun(
                          vmIp : "${params.vmIp}",
                          itestPath : "itests/restapi/dps",
                          itestName : "DPS",
                          additionalArgs : "-Prun-its,run-its-irvine"
                          )
                  }
              }
          }

          Thuy Do added a comment - michaelneale I attached the pipeline view in my comment.  Do you need the groovy script I use?  If so, I will include in the new feature request as suggested by jamesdumay . Here is a snippet of a stage in my pipeline.  Other stages are similar. stage("dps") {     when {         expression { return params.enableDps }     }     steps {         timeout(time: 10, unit: 'HOURS') {             ecdmItestRun(                 vmIp : "${params.vmIp}",                 itestPath : "itests/restapi/dps",                 itestName : "DPS",                 additionalArgs : "-Prun-its,run-its-irvine"                 )         }     } }

          Ruud P added a comment -

          Is this working also for Parallel stages on different nodes?  I am facing at this moment some strange behavior,

          if  one of the when statements is false:

          Blue ocean shows "Queued Waiting for next available executor on ....." until the end of the complete parallel execution,  but i see in the console everything is running

          If all when statements are true i dont see anny issue

          below a small example code

           parallel {

                  stage(A') {
                    when {expression {params.A}}
                      steps {node(label: 'A') {bla bla }}}
                  stage('Bt') {
                    when {expression {params.B}}
                      steps {node(label: 'B') {bla bla}}}
                  stage('C') {
                    when {expression {params.C}}
                      steps {node(label: 'C') {bla bla}}}

          }

           

          Ruud P added a comment - Is this working also for Parallel stages on different nodes?  I am facing at this moment some strange behavior, if  one of the when statements is false: Blue ocean shows "Queued Waiting for next available executor on ....." until the end of the complete parallel execution,  but i see in the console everything is running If all when statements are true i dont see anny issue below a small example code  parallel {         stage(A') {           when {expression {params.A}}             steps {node(label: 'A') {bla bla }}}         stage('Bt') {           when {expression {params.B}}             steps {node(label: 'B') {bla bla}}}         stage('C') {           when {expression {params.C}}             steps {node(label: 'C') {bla bla}}} }  

          Michael Neale added a comment -

          ruudp I don't think it works in parallel yet (feel free to open a new ticket though)

          Michael Neale added a comment - ruudp I don't think it works in parallel yet (feel free to open a new ticket though)

          Nat Sr added a comment -

          jamesdumay sophistifunk Hi guys, I wonder instead of adding "when{....}" to determine whether the stage will be skipped or not, can we have user input feature to ask if they want to skip the stage or not. Do we have this feature too?

          Nat Sr added a comment - jamesdumay sophistifunk Hi guys, I wonder instead of adding "when{....}" to determine whether the stage will be skipped or not, can we have user input feature to ask if they want to skip the stage or not. Do we have this feature too?

          Josh McDonald added a comment -

          I imagine you'd achieve that easily enough with an input step to set a variable?

          Josh McDonald added a comment - I imagine you'd achieve that easily enough with an input step to set a variable?

          Michael Neale added a comment -

          comscience15 not explicitly no - but like Josh says, could have an input step in an earlier stage that captures some environment variable that is then referred to in 'when'. A parametrised job could also provide the same variable I think. Can't put an input inside a 'when' (I think). 

          Michael Neale added a comment - comscience15 not explicitly no - but like Josh says, could have an input step in an earlier stage that captures some environment variable that is then referred to in 'when'. A parametrised job could also provide the same variable I think. Can't put an input inside a 'when' (I think). 

          Ruud P added a comment -

          it is fixed with an update of jenkins, at this moment this works fine for me

          Ruud P added a comment - it is fixed with an update of jenkins, at this moment this works fine for me

          Nat Sr added a comment -

          sophistifunk michaelneale Thanks for your feedback. Yes, it is easy to set a variable. My thought is just about if there is a complicated case which is sometimes is outside the set up variable. So if we can get it with user input too, it would be great. 

          Nat Sr added a comment - sophistifunk michaelneale Thanks for your feedback. Yes, it is easy to set a variable. My thought is just about if there is a complicated case which is sometimes is outside the set up variable. So if we can get it with user input too, it would be great. 

          Lars Nielsen added a comment -

          The graph in scripted pipelines does not show the line flowing around the skipped stage, but through it, like executed steps.

          The steps are shown as "No steps" and the full job log is displayed.

          Lars Nielsen added a comment - The graph in scripted pipelines does not show the line flowing around the skipped stage, but through it, like executed steps. The steps are shown as "No steps" and the full job log is displayed.

            sophistifunk Josh McDonald
            jamesdumay James Dumay
            Votes:
            10 Vote for this issue
            Watchers:
            21 Start watching this issue

              Created:
              Updated:
              Resolved: