Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-47799

Blue Ocean shouldn't visualize the parallel keyword of a scripted pipeline directly

    • Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Major Major
    • blueocean-plugin
    • Jenkins 3.73.2, Blue Ocean Plugin 1.3.1, Pipeline 2.5

      Problem
      Sophisticated users of Pipeline want a more usable way to visualize and interact with their Pipelines. Blue Ocean cannot visualise pipelines in a meaningful way to users where there are a large number of stages or parallels.

      Goals

      1. Reduce unnecessarily visualised parallel branches by reserving parallel syntaxes for visualisation and non-visualisation purposes.
      2. Increase developer satisfaction by making the rules of Pipeline visualisation more consistent and the visualisation usable at any scale.

      See Rationalising Parallel.

      Original Request
      I believe the Pipeline keyword parallel should not be rendered by Ocean View directly respectively in it's on right , but only with transitive dependent Stages. Exception is when there is no stage enclosed by the parallel keyword. Background: since parallel can be used with enclosed stages and not only for the parallel execution of steps, the keyword has moved from being a mere implementation tool to a conceptional one for modeling workflows, this needs to be reflected in the visualization.  In the following comments there are some examples, which try to proove the point respectively my expectations.

      I am aware that there maybe nesting limitations, respectively how deeply the parallel keyword resp. stages are nested.  Here i pratical tread-offs should be made.

        1. ex1.png
          ex1.png
          26 kB
        2. ex2.png
          ex2.png
          24 kB
        3. ex3.png
          ex3.png
          25 kB
        4. ex4.png
          ex4.png
          41 kB

          [JENKINS-47799] Blue Ocean shouldn't visualize the parallel keyword of a scripted pipeline directly

          Example: 

           

          stage("Stage A" ) {
          node { echo "Task of A"}
          } 
          parallel ('Parallel B': {
          stage ("Stage B") { node { echo "Task of Stage B" 
          echo "Would expect Stage B instead of Parallel B"
          } }
          }, 'Parallel C':{
          stage ("Stage C"){node { echo "Task of Stage C"}
          echo "Would expect Stage C instead of Parallel C"
          }
          })
          stage("Stage D" ) {
          node { echo "Task of D"}
          }
          

          Blue Ocean show here see attachment ex2.png

          Here i would clearly expect Stage B and Stage C instead of Parallel B and Parallel C.

           

           

          Christoph Henrici added a comment - Example:    stage( "Stage A" ) { node { echo "Task of A" } } parallel ( 'Parallel B' : { stage ( "Stage B" ) { node { echo "Task of Stage B" echo "Would expect Stage B instead of Parallel B" } } }, 'Parallel C' :{ stage ( "Stage C" ){node { echo "Task of Stage C" } echo "Would expect Stage C instead of Parallel C" } }) stage( "Stage D" ) { node { echo "Task of D" } } Blue Ocean show here see attachment ex2.png Here i would clearly expect Stage B and Stage C instead of Parallel B and Parallel C.    

          Christoph Henrici added a comment - - edited

          Example : 

           

          stage("Stage A" ) {
          node { echo "Task of Stage A"}
          }
          parallel ('Parallel B': {
          stage ("Stage B") { node { echo "Task of Stage B"
          echo "Would at least expect to see Stage B Parallel to Stage C"
          } }
          }, 'Parallel C':{
          stage ("Stage C"){
          stage ("Stage C.1") {node { echo "Task of Stage C.1"
          echo "Would at least expect to see Stage C Parallel to Stage B, if not Stage C.2 in sequence to C.1"
          }
          } 
          stage ("Stage C.2 ") {node { echo "Task of Stage C.2"
          echo "Would at least expect to see Stage C Parallel to Stage B, if not Stage C.2 in sequence to C.1"
          }
          } 
          }
          })
          stage("Stage D" ) {
          node { echo "Task of Stage D"}
          }
          

          Blue Ocean show here see attachment ex3.png

          Would at least expect to see Stage C Parallel to Stage B, if not Stage C.2 in sequence to C.1 parallel to B. For me the fact that Parallel B and Parallel C are shown is clearly a conceptional break with regard to Stage A and Stage B: the keyword parallel and stages are two completely different conceptional entities. 

           

          Christoph Henrici added a comment - - edited Example :    stage( "Stage A" ) { node { echo "Task of Stage A" } } parallel ( 'Parallel B' : { stage ( "Stage B" ) { node { echo "Task of Stage B" echo "Would at least expect to see Stage B Parallel to Stage C" } } }, 'Parallel C' :{ stage ( "Stage C" ){ stage ( "Stage C.1" ) {node { echo "Task of Stage C.1" echo "Would at least expect to see Stage C Parallel to Stage B, if not Stage C.2 in sequence to C.1" } } stage ( "Stage C.2 " ) {node { echo "Task of Stage C.2" echo "Would at least expect to see Stage C Parallel to Stage B, if not Stage C.2 in sequence to C.1" } } } }) stage( "Stage D" ) { node { echo "Task of Stage D" } } Blue Ocean show here see attachment ex3.png Would at least expect to see Stage C Parallel to Stage B, if not Stage C.2 in sequence to C.1 parallel to B. For me the fact that Parallel B and Parallel C are shown is clearly a conceptional break with regard to Stage A and Stage B: the keyword parallel and stages are two completely different conceptional entities.   

          Christoph Henrici added a comment - - edited

          Somewhat more complicated : 

          stage ("Stage A") { node { echo "Task of Stage A"} }
          parallel ('Parallel A ': {
          stage ("Stage A") { node { echo "Task of Stage A"
          echo "Expect to see Stage A in Parallel with Stage B"
          } 
          }
          }, 'Parallel B':{
          stage ("Stage B"){
          stage ("Stage B.1 ") {node { echo "Task of Stage B.1.1"
          echo "Expect to see Stage A in Parallel with Stage B at least"
          echo "Possibly Stage A in Parallel with a Sequence of B.1, B.2, B.3"
          }} 
          stage ("Stage B.2") {node { echo "Task of Stage B.1,2"
          echo "Expect to see Stage A in Parallel with Stage B at least"
          echo "Possibly Stage A in Parallel with a Sequence of B.1, B.2, B.3"
          }} 
          stage("Stage B.3") {
          parallel ('Parallel B.3.1':{
          node { echo "Task of Stage B.3, Parallel B.3.1"
          echo "Expect to see Stage A in Parallel with Stage B at least"
          echo "Possibly Stage A in Parallel with a Sequence of B.1, B.2, B.3"
          echo "Possibly Stage A in Parallel with a Sequence of B.1, B.2 in Parallel with a sequence of B.3.1, B.3.2 "
          }
          }, 'Parallel B.3.2':{
          node { echo "Task of Stage B.3, Parallel B.3.2"
          echo "Expect to see Stage A in Parallel with Stage B at least"
          echo "Possibly Stage A in Parallel with a Sequence of B.1, B.2, B.3"
          echo "Possibly Stage A in Parallel with a Sequence of B.1, B.2 in Parallel with a sequence of B.3.1, B.3.2 "
          }
          }) 
          }
          }
          stage ("Stage C") { node { echo "Task of Stage C"} }
          
          })
          stage("Stage D" ) {
          node { echo "Task of Stage d"}
          }
          

          See attachment ex4.png

           Here the conceptional break of visualizing stages and parallel is even clearer from my point of view. At the least i would expect to see Stage A in parallel with Stage B. The visualization of  Parallel A being parallel with B.3.1 and B.3.2 is misleading, resp. is pure coincidence if it is. 

           

          Christoph Henrici added a comment - - edited Somewhat more complicated :  stage ( "Stage A" ) { node { echo "Task of Stage A" } } parallel ( 'Parallel A ' : { stage ( "Stage A" ) { node { echo "Task of Stage A" echo "Expect to see Stage A in Parallel with Stage B" } } }, 'Parallel B' :{ stage ( "Stage B" ){ stage ( "Stage B.1 " ) {node { echo "Task of Stage B.1.1" echo "Expect to see Stage A in Parallel with Stage B at least" echo "Possibly Stage A in Parallel with a Sequence of B.1, B.2, B.3" }} stage ( "Stage B.2" ) {node { echo "Task of Stage B.1,2" echo "Expect to see Stage A in Parallel with Stage B at least" echo "Possibly Stage A in Parallel with a Sequence of B.1, B.2, B.3" }} stage( "Stage B.3" ) { parallel ( 'Parallel B.3.1' :{ node { echo "Task of Stage B.3, Parallel B.3.1" echo "Expect to see Stage A in Parallel with Stage B at least" echo "Possibly Stage A in Parallel with a Sequence of B.1, B.2, B.3" echo "Possibly Stage A in Parallel with a Sequence of B.1, B.2 in Parallel with a sequence of B.3.1, B.3.2 " } }, 'Parallel B.3.2' :{ node { echo "Task of Stage B.3, Parallel B.3.2" echo "Expect to see Stage A in Parallel with Stage B at least" echo "Possibly Stage A in Parallel with a Sequence of B.1, B.2, B.3" echo "Possibly Stage A in Parallel with a Sequence of B.1, B.2 in Parallel with a sequence of B.3.1, B.3.2 " } }) } } stage ( "Stage C" ) { node { echo "Task of Stage C" } } }) stage( "Stage D" ) { node { echo "Task of Stage d" } } See attachment ex4.png  Here the conceptional break of visualizing stages and parallel is even clearer from my point of view. At the least i would expect to see Stage A in parallel with Stage B. The visualization of  Parallel A being parallel with B.3.1 and B.3.2 is misleading, resp. is pure coincidence if it is.   

          James Dumay added a comment -

          Hi chenrici,

          I had a very similar thought a few months ago. I pulled this document out of our internal wiki and made it publicly available.

          Let me know what you think.

          Thanks,
          James

          James Dumay added a comment - Hi chenrici , I had a very similar thought a few months ago. I pulled this document out of our internal wiki and made it publicly available. Let me know what you think. Thanks, James

          Michael Neale added a comment -

          very interesting chenrici

          Michael Neale added a comment - very interesting chenrici

          jamesdumay that's a very valid proposal. If i understand correctly the key word parallel would be in the long run ignored in Blue Ocean visualization except for before stages also for scripted pipelines. 

          "Parallel execution is used to speed up the execution of the pipeline by having different parts run in parallel. They have little or no bearing on the stage flow graph and in generally not useful to visualise." Absolutely agreed. 

          And with this approach , if a developer finds parallel processing important to visualize he can use the proposed syntax

           

          Christoph Henrici added a comment - jamesdumay that's  a very valid proposal. If i understand correctly the key word parallel would be in the long run ignored in Blue Ocean visualization except for before stages also for scripted pipelines.  "Parallel execution is used to speed up the execution of the pipeline by having different parts run in parallel. They have little or no bearing on the stage flow graph and in generally not useful to visualise." Absolutely agreed.  And with this approach , if a developer finds parallel processing important to visualize he can use the proposed syntax  

          James Dumay added a comment -

          chenrici great to hear that you concur!

          James Dumay added a comment - chenrici great to hear that you concur!

          Jesse Glick added a comment -

          Parallel Stage syntax from Declarative is made available for scripted Pipelines.

          does not make sense. Scripted is a superset of Declarative so of course you can already decide in a scripted Pipeline whether to introduce a stage in each branch or not. No changes to the parallel step are required or advisable.

          Jesse Glick added a comment - Parallel Stage syntax from Declarative is made available for scripted Pipelines. does not make sense. Scripted is a superset of Declarative so of course you can already decide in a scripted Pipeline whether to introduce a stage in each branch or not. No changes to the parallel step are required or advisable.

          Sam Van Oort added a comment -

          This doesn't make sense to me – the parallel construct is heavily used and works in its intended role.  What isn't fully supported is nesting, but parallel branches are labelled for a reason. 

          The better solution is to fully support nesting, or ensure that for Scripted pipelines Blue Ocean does the same hack on Parallels with nested Stages that it does with Declarative. 

          Better yet, let the UI support nesting fully, but with the UX showing "Parallel A { Stage A {" as one branch if there are no other stages/parallels nested inside the parallel (to reduce visual clutter).   The lower-level APIs are all written to permit nesting of any stage/parallel combo – it just isn't propagated up to the UI because that lacks support.

          Sam Van Oort added a comment - This doesn't make sense to me – the parallel construct is heavily used and works in its intended role.  What isn't fully supported is nesting, but parallel branches are labelled for a reason.  The better solution is to fully support nesting, or ensure that for Scripted pipelines Blue Ocean does the same hack on Parallels with nested Stages that it does with Declarative.  Better yet, let the UI support nesting fully, but with the UX showing "Parallel A { Stage A {" as one branch if there are no other stages/parallels nested inside the parallel (to reduce visual clutter).   The lower-level APIs are all written to permit nesting of any stage/parallel combo – it just isn't propagated up to the UI because that lacks support.

          Christoph Henrici added a comment - - edited

          I see justification for both argumentations of svanoort and jamesdumay. The difference is a matter of emphasis, priorisations and feasibility. The main point being though what you want to vizualize. And here i my opinion is unchanged. Parallel is a adjective:  it's  parallel somethings ,  parallel stages , parallel steps , parallel tasks,  parallel processes, whatever... so you want to vizualize stages, steps as node and additionally the fact that they are parallel in ralation to other stages , tasks. You do NOT what to vizualize parallel as a node itself, which is want currently is happening in Ocean View. 

          Christoph Henrici added a comment - - edited I see justification for both argumentations of svanoort and jamesdumay . The difference is a matter of emphasis, priorisations and feasibility. The main point being though what you want to vizualize. And here i my opinion is unchanged. Parallel is a adjective:  it's  parallel somethings ,  parallel stages , parallel steps , parallel tasks,  parallel processes, whatever... so you want to vizualize stages, steps as node and additionally the fact that they are parallel in ralation to other stages , tasks. You do NOT what to vizualize parallel as a node itself, which is want currently is happening in Ocean View. 

            Unassigned Unassigned
            chenrici Christoph Henrici
            Votes:
            6 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated: