Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-47799

Blue Ocean shouldn't visualize the parallel keyword of a scripted pipeline directly

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      Problem
      Sophisticated users of Pipeline want a more usable way to visualize and interact with their Pipelines. Blue Ocean cannot visualise pipelines in a meaningful way to users where there are a large number of stages or parallels.

      Goals

      1. Reduce unnecessarily visualised parallel branches by reserving parallel syntaxes for visualisation and non-visualisation purposes.
      2. Increase developer satisfaction by making the rules of Pipeline visualisation more consistent and the visualisation usable at any scale.

      See Rationalising Parallel.

      Original Request
      I believe the Pipeline keyword parallel should not be rendered by Ocean View directly respectively in it's on right , but only with transitive dependent Stages. Exception is when there is no stage enclosed by the parallel keyword. Background: since parallel can be used with enclosed stages and not only for the parallel execution of steps, the keyword has moved from being a mere implementation tool to a conceptional one for modeling workflows, this needs to be reflected in the visualization.  In the following comments there are some examples, which try to proove the point respectively my expectations.

      I am aware that there maybe nesting limitations, respectively how deeply the parallel keyword resp. stages are nested.  Here i pratical tread-offs should be made.

        Attachments

        1. ex1.png
          ex1.png
          26 kB
        2. ex2.png
          ex2.png
          24 kB
        3. ex3.png
          ex3.png
          25 kB
        4. ex4.png
          ex4.png
          41 kB

          Issue Links

            Activity

            chenrici Christoph Henrici created issue -
            chenrici Christoph Henrici made changes -
            Field Original Value New Value
            Attachment Bildschirmfoto 2017-11-02 um 22.03.54.png [ 40207 ]
            chenrici Christoph Henrici made changes -
            Attachment Senario1.png [ 40208 ]
            Hide
            chenrici Christoph Henrici added a comment - - edited

            Example: 

             

            stage("Stage A" ) {
            node { echo "Task of A"}
            }
            stage("Stage B") {
            parallel ('Parallel B.1':{
            node { echo "Step of Parallel B.1"
            echo "Expect only Stage B or ok, since parallel is not wrapped in a stage Parallel B.1"
            }
            }, 'Parallel B.2':{
            node { echo "Step of Parallel B.2"
            echo "Expect only Stage B or ok, since parallel is not wrapped in a stage Parallel B.2"
            }
            
            })
            }
            stage("Stage C" ) {
            node { echo "Task of C"}
            }
            

            Blue  Ocean show's see attachment ex1.png

            I would be ok with only Stage B, but since not stage word is enclosed ok.

             

             

             

            Show
            chenrici Christoph Henrici added a comment - - edited Example:    stage( "Stage A" ) { node { echo "Task of A" } } stage( "Stage B" ) { parallel ( 'Parallel B.1' :{ node { echo "Step of Parallel B.1" echo "Expect only Stage B or ok, since parallel is not wrapped in a stage Parallel B.1" } }, 'Parallel B.2' :{ node { echo "Step of Parallel B.2" echo "Expect only Stage B or ok, since parallel is not wrapped in a stage Parallel B.2" } }) } stage( "Stage C" ) { node { echo "Task of C" } } Blue  Ocean show's see attachment ex1.png I would be ok with only Stage B, but since not stage word is enclosed ok.      
            chenrici Christoph Henrici made changes -
            Attachment Bildschirmfoto 2017-11-02 um 22.03.54.png [ 40207 ]
            chenrici Christoph Henrici made changes -
            Attachment Senario1.png [ 40208 ]
            chenrici Christoph Henrici made changes -
            Attachment ex1.png [ 40209 ]
            chenrici Christoph Henrici made changes -
            Description I believe the Pipeline keyword parallel should not be rendered by Ocean View directly respectively in it's on right , but only with transitive dependent Stages. Exception is when there is no stage enclosed by the parallel keyword. Background: since parallel can be used with enclosed stages and not only for the parallel execution of steps, the keyword has moved from being a mere implementation tool to a conceptional one for modeling workflows, this needs to be reflected in the visualization.  In the following comments there are some examples, which try to proove the point respectively my expectations. I believe the Pipeline keyword parallel should not be rendered by Ocean View directly respectively in it's on right , but only with transitive dependent Stages. Exception is when there is no stage enclosed by the parallel keyword. Background: since parallel can be used with enclosed stages and not only for the parallel execution of steps, the keyword has moved from being a mere implementation tool to a conceptional one for modeling workflows, this needs to be reflected in the visualization.  In the following comments there are some examples, which try to proove the point respectively my expectations.

            I am aware that there maybe nesting limitations, respectively how deeply the parallel keyword resp. stages are nested.  Here i can except consistent limitations..
            Hide
            chenrici Christoph Henrici added a comment -

            Example: 

             

            stage("Stage A" ) {
            node { echo "Task of A"}
            } 
            parallel ('Parallel B': {
            stage ("Stage B") { node { echo "Task of Stage B" 
            echo "Would expect Stage B instead of Parallel B"
            } }
            }, 'Parallel C':{
            stage ("Stage C"){node { echo "Task of Stage C"}
            echo "Would expect Stage C instead of Parallel C"
            }
            })
            stage("Stage D" ) {
            node { echo "Task of D"}
            }
            

            Blue Ocean show here see attachment ex2.png

            Here i would clearly expect Stage B and Stage C instead of Parallel B and Parallel C.

             

             

            Show
            chenrici Christoph Henrici added a comment - Example:    stage( "Stage A" ) { node { echo "Task of A" } } parallel ( 'Parallel B' : { stage ( "Stage B" ) { node { echo "Task of Stage B" echo "Would expect Stage B instead of Parallel B" } } }, 'Parallel C' :{ stage ( "Stage C" ){node { echo "Task of Stage C" } echo "Would expect Stage C instead of Parallel C" } }) stage( "Stage D" ) { node { echo "Task of D" } } Blue Ocean show here see attachment ex2.png Here i would clearly expect Stage B and Stage C instead of Parallel B and Parallel C.    
            chenrici Christoph Henrici made changes -
            Attachment ex2.png [ 40210 ]
            Hide
            chenrici Christoph Henrici added a comment - - edited

            Example : 

             

            stage("Stage A" ) {
            node { echo "Task of Stage A"}
            }
            parallel ('Parallel B': {
            stage ("Stage B") { node { echo "Task of Stage B"
            echo "Would at least expect to see Stage B Parallel to Stage C"
            } }
            }, 'Parallel C':{
            stage ("Stage C"){
            stage ("Stage C.1") {node { echo "Task of Stage C.1"
            echo "Would at least expect to see Stage C Parallel to Stage B, if not Stage C.2 in sequence to C.1"
            }
            } 
            stage ("Stage C.2 ") {node { echo "Task of Stage C.2"
            echo "Would at least expect to see Stage C Parallel to Stage B, if not Stage C.2 in sequence to C.1"
            }
            } 
            }
            })
            stage("Stage D" ) {
            node { echo "Task of Stage D"}
            }
            

            Blue Ocean show here see attachment ex3.png

            Would at least expect to see Stage C Parallel to Stage B, if not Stage C.2 in sequence to C.1 parallel to B. For me the fact that Parallel B and Parallel C are shown is clearly a conceptional break with regard to Stage A and Stage B: the keyword parallel and stages are two completely different conceptional entities. 

             

            Show
            chenrici Christoph Henrici added a comment - - edited Example :    stage( "Stage A" ) { node { echo "Task of Stage A" } } parallel ( 'Parallel B' : { stage ( "Stage B" ) { node { echo "Task of Stage B" echo "Would at least expect to see Stage B Parallel to Stage C" } } }, 'Parallel C' :{ stage ( "Stage C" ){ stage ( "Stage C.1" ) {node { echo "Task of Stage C.1" echo "Would at least expect to see Stage C Parallel to Stage B, if not Stage C.2 in sequence to C.1" } } stage ( "Stage C.2 " ) {node { echo "Task of Stage C.2" echo "Would at least expect to see Stage C Parallel to Stage B, if not Stage C.2 in sequence to C.1" } } } }) stage( "Stage D" ) { node { echo "Task of Stage D" } } Blue Ocean show here see attachment ex3.png Would at least expect to see Stage C Parallel to Stage B, if not Stage C.2 in sequence to C.1 parallel to B. For me the fact that Parallel B and Parallel C are shown is clearly a conceptional break with regard to Stage A and Stage B: the keyword parallel and stages are two completely different conceptional entities.   
            chenrici Christoph Henrici made changes -
            Attachment ex3.png [ 40211 ]
            Hide
            chenrici Christoph Henrici added a comment - - edited

            Somewhat more complicated : 

            stage ("Stage A") { node { echo "Task of Stage A"} }
            parallel ('Parallel A ': {
            stage ("Stage A") { node { echo "Task of Stage A"
            echo "Expect to see Stage A in Parallel with Stage B"
            } 
            }
            }, 'Parallel B':{
            stage ("Stage B"){
            stage ("Stage B.1 ") {node { echo "Task of Stage B.1.1"
            echo "Expect to see Stage A in Parallel with Stage B at least"
            echo "Possibly Stage A in Parallel with a Sequence of B.1, B.2, B.3"
            }} 
            stage ("Stage B.2") {node { echo "Task of Stage B.1,2"
            echo "Expect to see Stage A in Parallel with Stage B at least"
            echo "Possibly Stage A in Parallel with a Sequence of B.1, B.2, B.3"
            }} 
            stage("Stage B.3") {
            parallel ('Parallel B.3.1':{
            node { echo "Task of Stage B.3, Parallel B.3.1"
            echo "Expect to see Stage A in Parallel with Stage B at least"
            echo "Possibly Stage A in Parallel with a Sequence of B.1, B.2, B.3"
            echo "Possibly Stage A in Parallel with a Sequence of B.1, B.2 in Parallel with a sequence of B.3.1, B.3.2 "
            }
            }, 'Parallel B.3.2':{
            node { echo "Task of Stage B.3, Parallel B.3.2"
            echo "Expect to see Stage A in Parallel with Stage B at least"
            echo "Possibly Stage A in Parallel with a Sequence of B.1, B.2, B.3"
            echo "Possibly Stage A in Parallel with a Sequence of B.1, B.2 in Parallel with a sequence of B.3.1, B.3.2 "
            }
            }) 
            }
            }
            stage ("Stage C") { node { echo "Task of Stage C"} }
            
            })
            stage("Stage D" ) {
            node { echo "Task of Stage d"}
            }
            

            See attachment ex4.png

             Here the conceptional break of visualizing stages and parallel is even clearer from my point of view. At the least i would expect to see Stage A in parallel with Stage B. The visualization of  Parallel A being parallel with B.3.1 and B.3.2 is misleading, resp. is pure coincidence if it is. 

             

            Show
            chenrici Christoph Henrici added a comment - - edited Somewhat more complicated :  stage ( "Stage A" ) { node { echo "Task of Stage A" } } parallel ( 'Parallel A ' : { stage ( "Stage A" ) { node { echo "Task of Stage A" echo "Expect to see Stage A in Parallel with Stage B" } } }, 'Parallel B' :{ stage ( "Stage B" ){ stage ( "Stage B.1 " ) {node { echo "Task of Stage B.1.1" echo "Expect to see Stage A in Parallel with Stage B at least" echo "Possibly Stage A in Parallel with a Sequence of B.1, B.2, B.3" }} stage ( "Stage B.2" ) {node { echo "Task of Stage B.1,2" echo "Expect to see Stage A in Parallel with Stage B at least" echo "Possibly Stage A in Parallel with a Sequence of B.1, B.2, B.3" }} stage( "Stage B.3" ) { parallel ( 'Parallel B.3.1' :{ node { echo "Task of Stage B.3, Parallel B.3.1" echo "Expect to see Stage A in Parallel with Stage B at least" echo "Possibly Stage A in Parallel with a Sequence of B.1, B.2, B.3" echo "Possibly Stage A in Parallel with a Sequence of B.1, B.2 in Parallel with a sequence of B.3.1, B.3.2 " } }, 'Parallel B.3.2' :{ node { echo "Task of Stage B.3, Parallel B.3.2" echo "Expect to see Stage A in Parallel with Stage B at least" echo "Possibly Stage A in Parallel with a Sequence of B.1, B.2, B.3" echo "Possibly Stage A in Parallel with a Sequence of B.1, B.2 in Parallel with a sequence of B.3.1, B.3.2 " } }) } } stage ( "Stage C" ) { node { echo "Task of Stage C" } } }) stage( "Stage D" ) { node { echo "Task of Stage d" } } See attachment ex4.png  Here the conceptional break of visualizing stages and parallel is even clearer from my point of view. At the least i would expect to see Stage A in parallel with Stage B. The visualization of  Parallel A being parallel with B.3.1 and B.3.2 is misleading, resp. is pure coincidence if it is.   
            chenrici Christoph Henrici made changes -
            Attachment ex4.png [ 40212 ]
            chenrici Christoph Henrici made changes -
            Environment Jenkins 3.73.2, Blue Ocean Plugin 1.3.1 Jenkins 3.73.2, Blue Ocean Plugin 1.3.1,
            chenrici Christoph Henrici made changes -
            Description I believe the Pipeline keyword parallel should not be rendered by Ocean View directly respectively in it's on right , but only with transitive dependent Stages. Exception is when there is no stage enclosed by the parallel keyword. Background: since parallel can be used with enclosed stages and not only for the parallel execution of steps, the keyword has moved from being a mere implementation tool to a conceptional one for modeling workflows, this needs to be reflected in the visualization.  In the following comments there are some examples, which try to proove the point respectively my expectations.

            I am aware that there maybe nesting limitations, respectively how deeply the parallel keyword resp. stages are nested.  Here i can except consistent limitations..
            I believe the Pipeline keyword parallel should not be rendered by Ocean View directly respectively in it's on right , but only with transitive dependent Stages. Exception is when there is no stage enclosed by the parallel keyword. Background: since parallel can be used with enclosed stages and not only for the parallel execution of steps, the keyword has moved from being a mere implementation tool to a conceptional one for modeling workflows, this needs to be reflected in the visualization.  In the following comments there are some examples, which try to proove the point respectively my expectations.

            I am aware that there maybe nesting limitations, respectively how deeply the parallel keyword resp. stages are nested.  Here i pratical tread-offs should be made.
            chenrici Christoph Henrici made changes -
            Environment Jenkins 3.73.2, Blue Ocean Plugin 1.3.1, Jenkins 3.73.2, Blue Ocean Plugin 1.3.1, Pipeline 2.5
            jamesdumay James Dumay made changes -
            Remote Link This issue links to "Page (Jenkins Wiki)" [ 18146 ]
            Hide
            jamesdumay James Dumay added a comment -

            Hi Christoph Henrici,

            I had a very similar thought a few months ago. I pulled this document out of our internal wiki and made it publicly available.

            Let me know what you think.

            Thanks,
            James

            Show
            jamesdumay James Dumay added a comment - Hi Christoph Henrici , I had a very similar thought a few months ago. I pulled this document out of our internal wiki and made it publicly available. Let me know what you think. Thanks, James
            jamesdumay James Dumay made changes -
            Remote Link This issue links to "Rationalising Parallel (Web Link)" [ 18148 ]
            jamesdumay James Dumay made changes -
            Description I believe the Pipeline keyword parallel should not be rendered by Ocean View directly respectively in it's on right , but only with transitive dependent Stages. Exception is when there is no stage enclosed by the parallel keyword. Background: since parallel can be used with enclosed stages and not only for the parallel execution of steps, the keyword has moved from being a mere implementation tool to a conceptional one for modeling workflows, this needs to be reflected in the visualization.  In the following comments there are some examples, which try to proove the point respectively my expectations.

            I am aware that there maybe nesting limitations, respectively how deeply the parallel keyword resp. stages are nested.  Here i pratical tread-offs should be made.
            *Problem*
            Sophisticated users of Pipeline want a more usable way to visualize and interact with their Pipelines. Blue Ocean cannot visualise pipelines in a meaningful way to users where there are a large number of stages or parallels.

            *Goals*
            # Reduce unnecessarily visualised parallel branches by reserving parallel syntaxes for visualisation and non-visualisation purposes.
            # Increase developer satisfaction by making the rules of Pipeline visualisation more consistent and the visualisation usable at any scale.

            See [Rationalising Parallel|https://wiki.jenkins.io/display/JENKINS/Rationalising+parallel].

            *Original Request*
            I believe the Pipeline keyword parallel should not be rendered by Ocean View directly respectively in it's on right , but only with transitive dependent Stages. Exception is when there is no stage enclosed by the parallel keyword. Background: since parallel can be used with enclosed stages and not only for the parallel execution of steps, the keyword has moved from being a mere implementation tool to a conceptional one for modeling workflows, this needs to be reflected in the visualization.  In the following comments there are some examples, which try to proove the point respectively my expectations.

            I am aware that there maybe nesting limitations, respectively how deeply the parallel keyword resp. stages are nested.  Here i pratical tread-offs should be made.
            jamesdumay James Dumay made changes -
            Epic Link JENKINS-43952 [ 181484 ]
            Hide
            michaelneale Michael Neale added a comment -

            very interesting Christoph Henrici

            Show
            michaelneale Michael Neale added a comment - very interesting Christoph Henrici
            Hide
            chenrici Christoph Henrici added a comment -

            James Dumay that's a very valid proposal. If i understand correctly the key word parallel would be in the long run ignored in Blue Ocean visualization except for before stages also for scripted pipelines. 

            "Parallel execution is used to speed up the execution of the pipeline by having different parts run in parallel. They have little or no bearing on the stage flow graph and in generally not useful to visualise." Absolutely agreed. 

            And with this approach , if a developer finds parallel processing important to visualize he can use the proposed syntax

             

            Show
            chenrici Christoph Henrici added a comment - James Dumay that's  a very valid proposal. If i understand correctly the key word parallel would be in the long run ignored in Blue Ocean visualization except for before stages also for scripted pipelines.  "Parallel execution is used to speed up the execution of the pipeline by having different parts run in parallel. They have little or no bearing on the stage flow graph and in generally not useful to visualise." Absolutely agreed.  And with this approach , if a developer finds parallel processing important to visualize he can use the proposed syntax  
            Hide
            jamesdumay James Dumay added a comment -

            Christoph Henrici great to hear that you concur!

            Show
            jamesdumay James Dumay added a comment - Christoph Henrici great to hear that you concur!
            Hide
            jglick Jesse Glick added a comment -

            Parallel Stage syntax from Declarative is made available for scripted Pipelines.

            does not make sense. Scripted is a superset of Declarative so of course you can already decide in a scripted Pipeline whether to introduce a stage in each branch or not. No changes to the parallel step are required or advisable.

            Show
            jglick Jesse Glick added a comment - Parallel Stage syntax from Declarative is made available for scripted Pipelines. does not make sense. Scripted is a superset of Declarative so of course you can already decide in a scripted Pipeline whether to introduce a stage in each branch or not. No changes to the parallel step are required or advisable.
            Hide
            svanoort Sam Van Oort added a comment -

            This doesn't make sense to me – the parallel construct is heavily used and works in its intended role.  What isn't fully supported is nesting, but parallel branches are labelled for a reason. 

            The better solution is to fully support nesting, or ensure that for Scripted pipelines Blue Ocean does the same hack on Parallels with nested Stages that it does with Declarative. 

            Better yet, let the UI support nesting fully, but with the UX showing "Parallel A { Stage A {" as one branch if there are no other stages/parallels nested inside the parallel (to reduce visual clutter).   The lower-level APIs are all written to permit nesting of any stage/parallel combo – it just isn't propagated up to the UI because that lacks support.

            Show
            svanoort Sam Van Oort added a comment - This doesn't make sense to me – the parallel construct is heavily used and works in its intended role.  What isn't fully supported is nesting, but parallel branches are labelled for a reason.  The better solution is to fully support nesting, or ensure that for Scripted pipelines Blue Ocean does the same hack on Parallels with nested Stages that it does with Declarative.  Better yet, let the UI support nesting fully, but with the UX showing "Parallel A { Stage A {" as one branch if there are no other stages/parallels nested inside the parallel (to reduce visual clutter).   The lower-level APIs are all written to permit nesting of any stage/parallel combo – it just isn't propagated up to the UI because that lacks support.
            jamesdumay James Dumay made changes -
            Link This issue is duplicated by JENKINS-44820 [ JENKINS-44820 ]
            Hide
            chenrici Christoph Henrici added a comment - - edited

            I see justification for both argumentations of Sam Van Oort and James Dumay. The difference is a matter of emphasis, priorisations and feasibility. The main point being though what you want to vizualize. And here i my opinion is unchanged. Parallel is a adjective:  it's  parallel somethings ,  parallel stages , parallel steps , parallel tasks,  parallel processes, whatever... so you want to vizualize stages, steps as node and additionally the fact that they are parallel in ralation to other stages , tasks. You do NOT what to vizualize parallel as a node itself, which is want currently is happening in Ocean View. 

            Show
            chenrici Christoph Henrici added a comment - - edited I see justification for both argumentations of Sam Van Oort and James Dumay . The difference is a matter of emphasis, priorisations and feasibility. The main point being though what you want to vizualize. And here i my opinion is unchanged. Parallel is a adjective:  it's  parallel somethings ,  parallel stages , parallel steps , parallel tasks,  parallel processes, whatever... so you want to vizualize stages, steps as node and additionally the fact that they are parallel in ralation to other stages , tasks. You do NOT what to vizualize parallel as a node itself, which is want currently is happening in Ocean View. 
            allanlewis_youview Allan Lewis made changes -
            Summary Ocean View shouldn't visualize the parallel keyword of a scripted pipeline directly Blue Ocean shouldn't visualize the parallel keyword of a scripted pipeline directly

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              chenrici Christoph Henrici
              Votes:
              6 Vote for this issue
              Watchers:
              10 Start watching this issue

                Dates

                Created:
                Updated: