Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-39839

Missing StandardChunkVisitor.parallelBranchEnd() and parallelEnd() events if there are nested parallels

      workflow-api plugin version 2.6.

      For the given pipeline script, secondLevelNestedBranch1 and secondLevelNestedBranch2 parallel branches are nested inside nestedBranch.

      There are missing StandardChunkVisitor.parallelEnd() and StandardChunkVisitor.parallelBranchEnd() events for nestedBranch. This causes computing incorrect DAG and status of individual branches. Blueocean extends StandardChunkVisitor to process events and create it's DAG.

      node {
         stage ('test') { 
           echo ('Testing'); 
           parallel nestedBranch: {
             echo 'nested Branch'
             stage('nestedBranchStage') {
               echo 'running nestedBranchStage'
               parallel secondLevelNestedBranch1: {
                 echo 'secondLevelNestedBranch1'
               },secondLevelNestedBranch2: {
                 echo 'secondLevelNestedBranch2'
               }
             }
           },
          failFast: false
         } 
      }
      

      If there are only one parallel branches in nested parallel then no parallelEnd or parallelBranchEnd event is fired. See the script below:

      node {
         stage ('test') { 
           echo ('Testing'); 
           parallel nestedBranch: {
             echo 'nested Branch'
             stage('nestedBranchStage') {
               echo 'running nestedBranchStage'
               parallel secondLevelNestedBranch1: {
                 echo 'secondLevelNestedBranch1'
               }
             }
           },
          failFast: false
         } 
      }
      

          [JENKINS-39839] Missing StandardChunkVisitor.parallelBranchEnd() and parallelEnd() events if there are nested parallels

          Vivek Pandey added a comment -

          Looks like, firing of parallel events are broken in general. Below script has two stages, each stage with couple parallels, parallelStart event is not called. The pipeline script below has s2.0 and s2.1 are still running so parallelBranchEnd is not fired, understandable, but I expect parallelStart should have been fired.

          I collect branches in parallelStart and associate them with relevant stage, reset states etc. so this breaks association of parallel branches with the stage it belongs to. See https://issues.jenkins-ci.org/browse/JENKINS-39847

          chunkEnd=> id: 39, name: End of Pipeline, function: End of Pipeline
          chunkEnd=> id: 35, name: Stage : Body : End, function: }
            StartNode: org.jenkinsci.plugins.workflow.cps.nodes.StepStartNode[id=24]
          
          parallelEnd=> id: 20, StartNode: 12, name: Execute in parallel : Start, function: parallel
          parallelBranchStart=> id: 31, name: Branch: S2.1, function: { (Branch: S2.1)
          parallelBranchStart=> id: 26, name: Branch: S2.0, function: { (Branch: S2.0)
          
          
          handleChunkDone=> id: 24, name: S2, function: { (S2)
          chunkStart=> id: 24, name: S2, function: { (S2)
          
          
          chunkEnd=> id: 21, name: Stage : Body : End, function: }
            StartNode: org.jenkinsci.plugins.workflow.cps.nodes.StepStartNode[id=6]
          
          parallelEnd=> id: 20, StartNode: 12, name: Execute in parallel : Start, function: parallel
          
          parallelBranchEnd=> id: 19, StartNode: 15, name: Branch: S1.1.2, function: { (Branch: S1.1.2)
          parallelBranchStart=> id: 15, name: Branch: S1.1.2, function: { (Branch: S1.1.2)
          
          parallelBranchEnd=> id: 18, StartNode: 14, name: Branch: S1.1.1, function: { (Branch: S1.1.1)
          parallelBranchStart=> id: 14, name: Branch: S1.1.1, function: { (Branch: S1.1.1)
          
          parallelStart=> id: 12, name: Execute in parallel : Start, function: parallel
            branch=> id: 14, name: Branch: S1.1.1, function: { (Branch: S1.1.1)
          
          parallelBranchStart=> id: 8, name: Branch: S1.0, function: { (Branch: S1.0)
          handleChunkDone=> id: 6, name: S1, function: { (S1)
          chunkStart=> id: 6, name: S1, function: { (S1)
          
          node {
            stage('S1') {  
              parallel 'S1.0': { sh('sleep 10') }
              
              parallel 'S1.1.1': { sh('sleep 10') }, 'S1.1.2': { sh('sleep 10') }
            }
              stage('S2') {  
              parallel 'S2.0': { sh('sleep 10') }
              
              parallel 'S2.1': { sh('sleep 10') }
            }
          }
          

          Vivek Pandey added a comment - Looks like, firing of parallel events are broken in general. Below script has two stages, each stage with couple parallels, parallelStart event is not called. The pipeline script below has s2.0 and s2.1 are still running so parallelBranchEnd is not fired, understandable, but I expect parallelStart should have been fired. I collect branches in parallelStart and associate them with relevant stage, reset states etc. so this breaks association of parallel branches with the stage it belongs to. See https://issues.jenkins-ci.org/browse/JENKINS-39847 chunkEnd=> id: 39, name: End of Pipeline, function: End of Pipeline chunkEnd=> id: 35, name: Stage : Body : End, function: } StartNode: org.jenkinsci.plugins.workflow.cps.nodes.StepStartNode[id=24] parallelEnd=> id: 20, StartNode: 12, name: Execute in parallel : Start, function: parallel parallelBranchStart=> id: 31, name: Branch: S2.1, function: { (Branch: S2.1) parallelBranchStart=> id: 26, name: Branch: S2.0, function: { (Branch: S2.0) handleChunkDone=> id: 24, name: S2, function: { (S2) chunkStart=> id: 24, name: S2, function: { (S2) chunkEnd=> id: 21, name: Stage : Body : End, function: } StartNode: org.jenkinsci.plugins.workflow.cps.nodes.StepStartNode[id=6] parallelEnd=> id: 20, StartNode: 12, name: Execute in parallel : Start, function: parallel parallelBranchEnd=> id: 19, StartNode: 15, name: Branch: S1.1.2, function: { (Branch: S1.1.2) parallelBranchStart=> id: 15, name: Branch: S1.1.2, function: { (Branch: S1.1.2) parallelBranchEnd=> id: 18, StartNode: 14, name: Branch: S1.1.1, function: { (Branch: S1.1.1) parallelBranchStart=> id: 14, name: Branch: S1.1.1, function: { (Branch: S1.1.1) parallelStart=> id: 12, name: Execute in parallel : Start, function: parallel branch=> id: 14, name: Branch: S1.1.1, function: { (Branch: S1.1.1) parallelBranchStart=> id: 8, name: Branch: S1.0, function: { (Branch: S1.0) handleChunkDone=> id: 6, name: S1, function: { (S1) chunkStart=> id: 6, name: S1, function: { (S1) node { stage( 'S1' ) { parallel 'S1.0' : { sh( 'sleep 10' ) } parallel 'S1.1.1' : { sh( 'sleep 10' ) }, 'S1.1.2' : { sh( 'sleep 10' ) } } stage( 'S2' ) { parallel 'S2.0' : { sh( 'sleep 10' ) } parallel 'S2.1' : { sh( 'sleep 10' ) } } }

          Sam Van Oort added a comment -

          vivek I probably discussed this in the phone at the time, but for the record: it's impossible to guarantee parallel/branch end events are always fired in the case where there is a parallel with only one branch, because:

          1. For an incomplete branch block (still running) you may not have an end block
          2. If there is not more than one current head node (current execution) you have no way to know it's inside a parallel

          This is best handled in your visitor by retaining nodes in case a parallel branch start is discovered (for incomplete flows at least).

          I can, however provide guarantees that the parallel starts are correctly handled (and they need to be), and am hardening tests for this case.

          Sam Van Oort added a comment - vivek I probably discussed this in the phone at the time, but for the record: it's impossible to guarantee parallel/branch end events are always fired in the case where there is a parallel with only one branch, because: For an incomplete branch block (still running) you may not have an end block If there is not more than one current head node (current execution) you have no way to know it's inside a parallel This is best handled in your visitor by retaining nodes in case a parallel branch start is discovered (for incomplete flows at least). I can, however provide guarantees that the parallel starts are correctly handled (and they need to be), and am hardening tests for this case.

          Sam Van Oort added a comment -

          vivek This is awaiting your review (along with other fixes) in https://github.com/jenkinsci/workflow-api-plugin/pull/33

          Sam Van Oort added a comment - vivek This is awaiting your review (along with other fixes) in https://github.com/jenkinsci/workflow-api-plugin/pull/33

          Code changed in jenkins
          User: Sam Van Oort
          Path:
          src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/ForkScanner.java
          src/test/java/org/jenkinsci/plugins/workflow/graphanalysis/ForkScannerTest.java
          http://jenkins-ci.org/commit/workflow-api-plugin/96e2c2ed99384a9c9f4fb80cc8f8876d9b225504
          Log:
          Fix a couple forkscanner testcase quirks, and put JENKINS-39839 to bed along with JENKINS-39841

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Sam Van Oort Path: src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/ForkScanner.java src/test/java/org/jenkinsci/plugins/workflow/graphanalysis/ForkScannerTest.java http://jenkins-ci.org/commit/workflow-api-plugin/96e2c2ed99384a9c9f4fb80cc8f8876d9b225504 Log: Fix a couple forkscanner testcase quirks, and put JENKINS-39839 to bed along with JENKINS-39841

          Sam Van Oort added a comment -

          Fixed with explicit test coverage in workflow-api 2.12

          Sam Van Oort added a comment - Fixed with explicit test coverage in workflow-api 2.12

            svanoort Sam Van Oort
            vivek Vivek Pandey
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: