Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-39464

Pipeline with parallel and no stages does not get visualised correctly

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • blueocean-plugin
    • None
    • tethys

      Problem
      If you have a parallel declaration, but no stages, in classic stage view nothing shows up (you just have to look at log). In blue ocean, you get a single node (stage) which shows only the first branch.
      Expected behavior: should show 3 parallel branches (ideally)

      Example

      parallel 'branch1': {
              node('node1') {
                  stage('Setup') {
                      checkout([details removed])
                  }
                  stage('Unit and Integration Tests') {
                      bat '"my command to execute tests"'
                  }
              }
      }, 'branch2': {
              node('node2') {
                  stage('Setup') {
                      checkout([details removed])
                  }
                  stage('Unit and Integration Tests') {
                      bat '"my command to execute tests"'
                  }
              }
      }
      

      Workaround
      Simply wrap the parallels in a stage and it will work correctly.

      stage('Build and Test') {
        parallel 'branch1': {
                node('node1') {
                    stage('Setup') {
                        checkout([details removed])
                    }
                    stage('Unit and Integration Tests') {
                        bat '"my command to execute tests"'
                    }
                }
        }, 'branch2': {
                node('node2') {
                    stage('Setup') {
                        checkout([details removed])
                    }
                    stage('Unit and Integration Tests') {
                        bat '"my command to execute tests"'
                    }
                }
        }
      }
      

      It will then be visualised like:

      Other examples

      Parallel defined between stages but not nested in its own stage

      node {
      	stage('Checkout') {
      		checkout scm
      	}
      }
      
      parallel linux: {
      	node('Linux') {
      		stage('Build') {
      			echo 'Build linux'
      		}
      		stage('Tests') {
      			echo 'Tests linux'
      		}
      		stage('Static analysis') {
      			echo 'Static analysis linux'
      		}
      	}
      },
      windows: {
      	node('Windows') {
      		stage('Build') {
      			echo 'Build windows'
      		}
      		stage('Tests') {
      			echo 'Tests windows'
      		}
      		stage('Static analysis') {
      			echo 'Static analysis windows'
      		}
      	}
      }
      

      Result looks like this:

          [JENKINS-39464] Pipeline with parallel and no stages does not get visualised correctly

          James Dumay added a comment - - edited

          jlpinardon absolutely - we can't render those stage's within the parallels (see JENKINS-38442).

          The reason why it doesn't show correctly after you change the structure of the Pipeline while executing is that we rely on the previous run to "preview" what the flow of the Pipeline will be. Unfortunately as Pipeline is a scripting language we can't read ahead the structure without executing it. The good news is that the new declarative syntax for Pipeline doesn't have this problem - we can read the Jenkinsfile like a config file

          If you run it a second time you shouldn't see the problem.

          James Dumay added a comment - - edited jlpinardon absolutely - we can't render those stage 's within the parallels (see JENKINS-38442 ). The reason why it doesn't show correctly after you change the structure of the Pipeline while executing is that we rely on the previous run to "preview" what the flow of the Pipeline will be. Unfortunately as Pipeline is a scripting language we can't read ahead the structure without executing it. The good news is that the new declarative syntax for Pipeline doesn't have this problem - we can read the Jenkinsfile like a config file If you run it a second time you shouldn't see the problem.

          jlpinardon added a comment -

          Thanks for the good news I will try this as soon as possible.
          But concerning the display while running,the bad news is that it is still broken after several run.

          jlpinardon added a comment - Thanks for the good news I will try this as soon as possible. But concerning the display while running,the bad news is that it is still broken after several run.

          James Dumay added a comment - - edited

          svanoort vivek would it be feasible to synthesise a stage around a parallel when it is not within a stage itself? We're starting to see more and more questions about why the visualization doesn't work (obviously not an issue with Declarative). I would like to get an idea of how feasible this is and a rough estimate.

          James Dumay added a comment - - edited svanoort vivek would it be feasible to synthesise a stage around a parallel when it is not within a stage itself? We're starting to see more and more questions about why the visualization doesn't work (obviously not an issue with Declarative). I would like to get an idea of how feasible this is and a rough estimate.

          Vivek Pandey added a comment -

          jamesdumay I will start with what I think is work involved or issues we need to solve and then make determination:

          • This virtual stage would collate all orphan (with no parent stage) consecutive top level parallels (with no stages in between)
          • What would be the display name or label of each virtual stage?
          • Current implementation is event based parsing API (bismuth), it sends events for all nodes. For stage it sends events that help compute stages, status, timing info etc.
          • * Computation of stage would require copying/collating status from underlying parallel
          • * Computation of timing will involve capture timing from underlying parallel blocks. This might be incorrect unless we capture all out-of-stage nodes (steps as well) and then compute timing info (duration, start and end)
          • Sythentic stage id generation. node id is generated during execution, its simple integer. I am hoping CpsFlowExecution.iota() might just do it but maybe there are edge cases where there are chances of id conflict - like pipeline is midway execution so id of synthetic stage could be reused later? Or we generate UUID for these nodes ourselves?

          svanoort are there other complications it might cause or better approach to handle it?

          Vivek Pandey added a comment - jamesdumay I will start with what I think is work involved or issues we need to solve and then make determination: This virtual stage would collate all orphan (with no parent stage) consecutive top level parallels (with no stages in between) What would be the display name or label of each virtual stage? Current implementation is event based parsing API (bismuth), it sends events for all nodes. For stage it sends events that help compute stages, status, timing info etc. * Computation of stage would require copying/collating status from underlying parallel * Computation of timing will involve capture timing from underlying parallel blocks. This might be incorrect unless we capture all out-of-stage nodes (steps as well) and then compute timing info (duration, start and end) Sythentic stage id generation. node id is generated during execution, its simple integer. I am hoping CpsFlowExecution.iota() might just do it but maybe there are edge cases where there are chances of id conflict - like pipeline is midway execution so id of synthetic stage could be reused later? Or we generate UUID for these nodes ourselves? svanoort are there other complications it might cause or better approach to handle it?

          James Dumay added a comment -

          vivek if we have to synthesise a stage I am happy with it being called "Parallel". Would it be possible to attach some data to the synthesised node so that we can show a warning in the UI about how it has been transformed? I'd like to educate users more about the usage of Stage.

          James Dumay added a comment - vivek if we have to synthesise a stage I am happy with it being called "Parallel". Would it be possible to attach some data to the synthesised node so that we can show a warning in the UI about how it has been transformed? I'd like to educate users more about the usage of Stage.

          James Dumay added a comment -

          This fix will be released in 1.0.0-b19 next week.

          James Dumay added a comment - This fix will be released in 1.0.0-b19 next week.

          Jakub Pawlinski added a comment - - edited

          Hi, running Blue Ocean 1.1.4 in declarative pipeline I can't make it work:

           
          stages {  stage('Parallel1') { 
              parallel 'branch1': {
                node('n1') { stage('Unit 1') { echo "1" }}{{} }}{
              }, 'branch2': {
                node('n2') { stage('Unit 2') { echo "2" }
          }}

          • Result: Starting with version 0.5, steps in a stage must be in a steps block
             
            stages {
              steps {
                parallel 'branch1': { 
                  node('n1') { stage('Unit 1') { echo "1" } } 
                }, 'branch2': {
                  node('n2') { stage('Unit 2') { echo "2" } }
            }}
          • Result: Invalid step "stage" used - not allowed in this context - The stage step cannot be used in Declarative Pipelines
            Running any of above outside of stages{} results with – Expected a stage exception
             
            I have created https://groups.google.com/forum/#!topic/jenkinsci-users/ObAZcvsiZA8 so hopefully will find workaround there
             

          Jakub Pawlinski added a comment - - edited Hi, running Blue Ocean 1.1.4 in declarative pipeline I can't make it work:   stages {   stage('Parallel1') {       parallel 'branch1': {       node('n1') {  stage('Unit 1') {  echo "1" }}{{} }}{     }, 'branch2': {       node('n2') {  stage('Unit 2') {  echo "2"  }  } }} Result: Starting with version 0.5, steps in a stage must be in a steps block   stages {   steps {     parallel 'branch1': {        node('n1') {  stage('Unit 1') {  echo "1"  }  }      }, 'branch2': {       node('n2') {  stage('Unit 2') {  echo "2"  }   } }} Result: Invalid step "stage" used - not allowed in this context - The stage step cannot be used in Declarative Pipelines Running any of above outside of stages{} results with – Expected a stage exception   I have created https://groups.google.com/forum/#!topic/jenkinsci-users/ObAZcvsiZA8  so hopefully will find workaround there  

          Hi, I'm running Blue Ocean 1.3.5 and am building the following:

          stages {
            stage('Build') {
              parallel {
                stage('Synthesis') {
                  ...
                }
                stage('Compilation') {
                  // Generate compilation matrix
                  ...
                  catchError {
                    parallel compilations
                  }
                }
              }
            }
          }
          

          I have 3 issues:

          1. All parallel stages are flattened into a long list. Preferably Synthesis and Compilation should split from the main pipeline, and then each of them should split into their own matrices.
          2. The stage that is failing is not the dot that is red. When clicking on the dot in the graph, I get the correct stage, but the color does not reflect the result. See the following screenshots.
          3. A minor issue: the stage name doesn't fit in the column. (but that's unrelated to here).

          Selected the red dot but the stage passed:

          Selected the green dot underneath it, which is the actual stage that failed.

          Tsvi Mostovicz added a comment - Hi, I'm running Blue Ocean 1.3.5 and am building the following: stages { stage( 'Build' ) { parallel { stage( 'Synthesis' ) { ... } stage( 'Compilation' ) { // Generate compilation matrix ... catchError { parallel compilations } } } } } I have 3 issues: All parallel stages are flattened into a long list. Preferably Synthesis and Compilation should split from the main pipeline, and then each of them should split into their own matrices. The stage that is failing is not the dot that is red. When clicking on the dot in the graph, I get the correct stage, but the color does not reflect the result. See the following screenshots. A minor issue: the stage name doesn't fit in the column. (but that's unrelated to here). Selected the red dot but the stage passed: Selected the green dot underneath it, which is the actual stage that failed.

          vivek
          I'm facing the same problem. Any progress here?

          See my complete issue described here: JENKINS-39464

          Taras Postument added a comment - vivek I'm facing the same problem. Any progress here? See my complete issue described here:  JENKINS-39464

          Rishi Thakkar added a comment -

          Why does the resolution to this issue say "Fixed" but the issue itself is still "In Porgress"?

          Rishi Thakkar added a comment - Why does the resolution to this issue say "Fixed" but the issue itself is still "In Porgress"?

            vivek Vivek Pandey
            michaelneale Michael Neale
            Votes:
            8 Vote for this issue
            Watchers:
            23 Start watching this issue

              Created:
              Updated:
              Resolved: