Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-46809

Allow sequential stages inside parallel in Declarative syntax

    XMLWordPrintable

Details

    • Pipeline - April 2018

    Description

      Currently with -JENKINS-41334-, we set one or more stages to be executed in parallel but no "branches". Meaning that we cannot parallelise a sequence of stages.

       

      Example:

      pipeline {
  agent none
  stages {
    stage('Parallel stuff') {
      parallel 'branch 1' : {
        // Sequencial stages
        stage('Branch 1 stage 1'){
            agent any
            steps {
                echo "In branch 1 stage 1"
            }
        }
        stage('Branch 1 stage 2'){
            agent none // With that kind of sequencial stage, we can change the agent to run on
            steps {
                sleep 30
            }
        }
      }, 'branch 2': { // Parallel execution
        stage('Branch 2 stage 1'){
            agent any
            steps {
                echo "In branch 2 stage 1"
                sleep 60
            }
        }
      }
    }
  }
}

      Blue ocean possible view:

      Attachments

        Issue Links

          is duplicated by

          New Feature - A new feature of the product, which has yet to be developed. JENKINS-43336 Allow locking multiple stages in declarative pipeline

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. JENKINS-47163 Use same workspace or node in multiple stages in pipeline

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. JENKINS-48422 Support parallel execution of sequence of stages

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. JENKINS-47532 Add possibility to run several stages 1-by-1 in a 'parallel' thread

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. JENKINS-39119 Can't lock nodes between stages

          • Major - Major loss of function.
          • Closed
          is related to

          Bug - A problem which impairs or prevents the functions of the product. JENKINS-52084 Sequential stages post section does not execute in the expected context

          • Major - Major loss of function.
          • Closed
          relates to

          New Feature - A new feature of the product, which has yet to be developed. JENKINS-39932 Support more arbitrary Pipeline definitions

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. JENKINS-40986 Matrix structure for Declarative Pipeline

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. JENKINS-52025 I can edit my sequential declarative pipeline stages in the pipeline visualization plugin

          • Major - Major loss of function.
          • Open

          New Feature - A new feature of the product, which has yet to be developed. JENKINS-49050 I can view my sequential declarative pipeline stages in the pipeline visualization plugin

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. JENKINS-39119 Can't lock nodes between stages

          • Major - Major loss of function.
          • Closed
          links to

          Web Link CloudBees Internal UX-724

          Web Link CloudBees Internal UX-727

          Web Link PR #227

          Activity

            Ascending order - Click to sort in descending order
            View 53 older comments
            adamvoss Adam Voss added a comment -

            jbriden One use reason is to better control the executing node.

            In my example I have a sequential stage where each stage in the sequence may execute on a different node.  One of those stages that I would like to split into two stages, but can only do so if they execute in the same workspace because the second stage will depend on the local output of the first.  If I could have a nested sequential stage, I could define the agent on the nested sequential stage each of its stages would then share that workspace.

            My exiting alternative would be to stash the outputs form the first stage and unstash them on the 2nd, which (untested) seems like a lot more overhead and complexity than I justify for splitting the stage (especially if I were to split it into more than 2 stages).

            adamvoss Adam Voss added a comment - jbriden One use reason is to better control the executing node. In my example I have a sequential stage where each stage in the sequence may execute on a different node.  One of those stages that I would like to split into two stages, but can only do so if they execute in the same workspace because the second stage will depend on the local output of the first.  If I could have a nested sequential stage, I could define the agent on the nested sequential stage each of its stages would then share that workspace. My exiting alternative would be to stash the outputs form the first stage and unstash them on the 2nd, which (untested) seems like a lot more overhead and complexity than I justify for splitting the stage (especially if I were to split it into more than 2 stages).
            olle71 Oliver Santschi added a comment -

            Does this example works for anyone? 

            Do I need a special version for a special plugin?

            olle71 Oliver Santschi added a comment - Does this example works for anyone?  Do I need a special version for a special plugin?
            sheeeng Leonard Lee added a comment -

            olle71, please see JENKINS-49050. Also, see this gist.

            sheeeng Leonard Lee added a comment - olle71 , please see JENKINS-49050 . Also, see this gist .
            horangs Hokwang Lee added a comment -

            Please be interested in this JENKINS-53127 related issue.

            horangs Hokwang Lee added a comment - Please be interested in this JENKINS-53127 related issue.
            nroose Nick Roosevelt added a comment -

            For what is is worth, I can't figure out if this was done, and if it was, when it was, and what version it is in or how to use it. And also, it seems like if we had the ability to have multiple pipelines for one push, that would help with all of this workload management. I would like to have nested parallelism as well as nested sequence.

             

            nroose Nick Roosevelt added a comment - For what is is worth, I can't figure out if this was done, and if it was, when it was, and what version it is in or how to use it. And also, it seems like if we had the ability to have multiple pipelines for one push, that would help with all of this workload management. I would like to have nested parallelism as well as nested sequence.  

            People

              abayer Andrew Bayer
              banst Bastien Arata
              Votes:
              66 Vote for this issue
              Watchers:
              103 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: