Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-46809

Allow sequential stages inside parallel in Declarative syntax

    XMLWordPrintable

Details

    • Pipeline - April 2018

    Description

      Currently with -JENKINS-41334-, we set one or more stages to be executed in parallel but no "branches". Meaning that we cannot parallelise a sequence of stages.

       

      Example:

      pipeline {
        agent none
        stages {
          stage('Parallel stuff') {
            parallel 'branch 1' : {
              // Sequencial stages
              stage('Branch 1 stage 1'){
                  agent any
                  steps {
                      echo "In branch 1 stage 1"
                  }
              }
              stage('Branch 1 stage 2'){
                  agent none // With that kind of sequencial stage, we can change the agent to run on
                  steps {
                      sleep 30
                  }
              }
            }, 'branch 2': { // Parallel execution
              stage('Branch 2 stage 1'){
                  agent any
                  steps {
                      echo "In branch 2 stage 1"
                      sleep 60
                  }
              }
            }
          }
        }
      }
      

      Blue ocean possible view:

      Attachments

        Issue Links

          Activity

            For what is is worth, I can't figure out if this was done, and if it was, when it was, and what version it is in or how to use it. And also, it seems like if we had the ability to have multiple pipelines for one push, that would help with all of this workload management. I would like to have nested parallelism as well as nested sequence.

             

            nroose Nick Roosevelt added a comment - For what is is worth, I can't figure out if this was done, and if it was, when it was, and what version it is in or how to use it. And also, it seems like if we had the ability to have multiple pipelines for one push, that would help with all of this workload management. I would like to have nested parallelism as well as nested sequence.  
            horangs Hokwang Lee added a comment -

            Please be interested in this JENKINS-53127 related issue.

            horangs Hokwang Lee added a comment - Please be interested in this JENKINS-53127 related issue.
            sheeeng Leonard Lee added a comment -

            olle71, please see JENKINS-49050. Also, see this gist.

            sheeeng Leonard Lee added a comment - olle71 , please see JENKINS-49050 . Also, see this gist .

            Does this example works for anyone? 

            Do I need a special version for a special plugin?

            olle71 Oliver Santschi added a comment - Does this example works for anyone?  Do I need a special version for a special plugin?
            adamvoss Adam Voss added a comment -

            jbriden One use reason is to better control the executing node.

            In my example I have a sequential stage where each stage in the sequence may execute on a different node.  One of those stages that I would like to split into two stages, but can only do so if they execute in the same workspace because the second stage will depend on the local output of the first.  If I could have a nested sequential stage, I could define the agent on the nested sequential stage each of its stages would then share that workspace.

            My exiting alternative would be to stash the outputs form the first stage and unstash them on the 2nd, which (untested) seems like a lot more overhead and complexity than I justify for splitting the stage (especially if I were to split it into more than 2 stages).

            adamvoss Adam Voss added a comment - jbriden One use reason is to better control the executing node. In my example I have a sequential stage where each stage in the sequence may execute on a different node.  One of those stages that I would like to split into two stages, but can only do so if they execute in the same workspace because the second stage will depend on the local output of the first.  If I could have a nested sequential stage, I could define the agent on the nested sequential stage each of its stages would then share that workspace. My exiting alternative would be to stash the outputs form the first stage and unstash them on the 2nd, which (untested) seems like a lot more overhead and complexity than I justify for splitting the stage (especially if I were to split it into more than 2 stages).

            People

              abayer Andrew Bayer
              banst Bastien Arata
              Votes:
              66 Vote for this issue
              Watchers:
              103 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: