Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-53032

Support dynamic definition of parallel running stages

      I would like to define parallel stages{} blocks that are executed in parallel dynamically during runtime.

      The stages{} blocks should run in parallel, the stage{} definitions in the parallel stages{} blocks should run sequential.

       

      A usecase for this is that the pipeline determines which applications changed in a branch and then creates parallel stages{} block to run build, checks, test etc per application.
      Per application tests should only run if the previous build stage for the app succeeded.

      It seems to be not supported currently. I couldn't find a way to realize it. I tried different variations of the syntax but couldn't make it work.

      Example of what I would like to do:

      pipeline {
        agent none
      
        stages {
          stage('Discover Changed Applications') {
            agent any
            steps {
              script {
                apps = ["app-one", "app-two"]
                }
              }
          }
      
          stage('parallel') {
            steps {
              script {
                pstages = [:]
      
                for (int i = 0; i < apps.size(); i++) {
                  app = apps[i]
      
                  pstages[app] = {
                    stages {
                      stage('Build application ') {
                        agent { label 'build' }
      
                        steps{
                          sh 'echo build'
                        }
                      }
                      stage('Test application ') {
                        agent { label 'test' }
      
                        steps{
                          sh 'echo test'
                        }
                      }
                    }
                  }
                }
      
                parallel(pstages)
              }
            }
          }
        }
      }
      

          [JENKINS-53032] Support dynamic definition of parallel running stages

          Steve Todorov added a comment - - edited

          I've answered a similar question at devops.stackexchange.com. I believe this is a bit overcomplicated and the Jenkins team should probably think of a way to simplify how dynamic stages (be they parallel or not) can be used in a declarative pipeline.

          Steve Todorov added a comment - - edited I've answered a similar question at devops.stackexchange.com . I believe this is a bit overcomplicated and the Jenkins team should probably think of a way to simplify how dynamic stages (be they parallel or not) can be used in a declarative pipeline.

          Ugur OZCAN added a comment -

          Steve's solution looks like a workaround we need a similar solution but we dont want to create a pipeline with groovy script.

          Why this is not a use case reading from a Listmap and using those values in parallel stages? I bet this is a very useful feature and valid usecase which tousands can benefit from.

          For example we have a device list and we can keep it as simple  [[name: AA, id:123][name:BB , id:234], ....] So as a listmap and we want to use device list (ids) each in different parallel stages like:

           

          Ugur OZCAN added a comment - Steve's solution looks like a workaround we need a similar solution but we dont want to create a pipeline with groovy script. Why this is not a use case reading from a Listmap and using those values in parallel stages? I bet this is a very useful feature and valid usecase which tousands can benefit from. For example we have a device list and we can keep it as simple  [ [name: AA, id:123] ,  [name:BB , id:234] , ....] So as a listmap and we want to use device list (ids) each in different parallel stages like:  

            Unassigned Unassigned
            fho Fabian Holler
            Votes:
            8 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated: