Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-55600

Parallel execution of same job N times with different parameters leads to only 1 execution of the job

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      1) I created sample job (see "sample_job_config.xml" attachment) which has "PROJECT_NAME" parameter
      2) I want to run it in parallel 5 times with different "PROJECT_NAME" values. I created following script for this:

      pipeline {
          agent none
          stages {
              stage('Processing projects') {
                  steps {
                      script {
                          def projects = ['project0', 'project1', 'project2', 'project3', 'project4']
                          def projectsBuilds = [:]
      
                          def labelParameters
                          for (project in projects) {
                              //JENKINS-55426 need to save iterated variable value into intermediate variable, otherwise there will be last value from collection for all jobs
                              def currentProject = project
                              labelParameters = []
                              labelParameters.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                              labelParameters.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: currentProject])
      
                              projectsBuilds[currentProject] = {
                                  stage(String.format('%s execution', currentProject)) {
                                      build job: 'Sample-Job', parameters: labelParameters
                                  }
                              }
                          }
      
                          parallel projectsBuilds;
                      }
                  }
              }
          }
      }
      

      3) However, when I run this script it only invokes job ONCE, not FIVE TIMES (see screen)

        Attachments

          Issue Links

            Activity

            alpanshin Alexandr Panshin created issue -
            alpanshin Alexandr Panshin made changes -
            Field Original Value New Value
            Description # I created sample job (see "sample_job_config.xml" attachment) which has "PROJECT_NAME" parameter
            # I want to run it in parallel 5 times with different "PROJECT_NAME" values. I created following script for this:

            {code:java}
            pipeline {
                agent none
                stages {
                    stage('Processing projects') {
                        steps {
                            script {
                                def projectsBuilds = [:]

                                def labelParameters0 = []
                                labelParameters0.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters0.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project0'])
                                projectsBuilds['project0'] = {
                                    stage(String.format('%s execution', 'project0')) {
                                        build job: 'Sample-Job', parameters: labelParameters0
                                    }
                                }

                                def labelParameters1 = []
                                labelParameters0.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters0.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project1'])
                                projectsBuilds['project1'] = {
                                    stage(String.format('%s execution', 'project1')) {
                                        build job: 'Sample-Job', parameters: labelParameters1
                                    }
                                }

                                def labelParameters2 = []
                                labelParameters0.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters0.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project2'])
                                projectsBuilds['project2'] = {
                                    stage(String.format('%s execution', 'project2')) {
                                        build job: 'Sample-Job', parameters: labelParameters2
                                    }
                                }

                                def labelParameters3 = []
                                labelParameters0.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters0.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project3'])
                                projectsBuilds['project3'] = {
                                    stage(String.format('%s execution', 'project3')) {
                                        build job: 'Sample-Job', parameters: labelParameters3
                                    }
                                }

                                def labelParameters4 = []
                                labelParameters0.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters0.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project4'])
                                projectsBuilds['project4'] = {
                                    stage(String.format('%s execution', 'project4')) {
                                        build job: 'Sample-Job', parameters: labelParameters4
                                    }
                                }

                                parallel projectsBuilds;
                            }
                        }
                    }
                }
            }
            {code}

            # However, when I run this script it only invokes job TWICE, not FIVE TIMES (see screens)
            1) I created sample job (see "sample_job_config.xml" attachment) which has "PROJECT_NAME" parameter
            2) I want to run it in parallel 5 times with different "PROJECT_NAME" values. I created following script for this:

            {code:java}
            pipeline {
                agent none
                stages {
                    stage('Processing projects') {
                        steps {
                            script {
                                def projectsBuilds = [:]

                                def labelParameters0 = []
                                labelParameters0.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters0.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project0'])
                                projectsBuilds['project0'] = {
                                    stage(String.format('%s execution', 'project0')) {
                                        build job: 'Sample-Job', parameters: labelParameters0
                                    }
                                }

                                def labelParameters1 = []
                                labelParameters0.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters0.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project1'])
                                projectsBuilds['project1'] = {
                                    stage(String.format('%s execution', 'project1')) {
                                        build job: 'Sample-Job', parameters: labelParameters1
                                    }
                                }

                                def labelParameters2 = []
                                labelParameters0.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters0.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project2'])
                                projectsBuilds['project2'] = {
                                    stage(String.format('%s execution', 'project2')) {
                                        build job: 'Sample-Job', parameters: labelParameters2
                                    }
                                }

                                def labelParameters3 = []
                                labelParameters0.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters0.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project3'])
                                projectsBuilds['project3'] = {
                                    stage(String.format('%s execution', 'project3')) {
                                        build job: 'Sample-Job', parameters: labelParameters3
                                    }
                                }

                                def labelParameters4 = []
                                labelParameters0.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters0.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project4'])
                                projectsBuilds['project4'] = {
                                    stage(String.format('%s execution', 'project4')) {
                                        build job: 'Sample-Job', parameters: labelParameters4
                                    }
                                }

                                parallel projectsBuilds;
                            }
                        }
                    }
                }
            }
            {code}

            3) However, when I run this script it only invokes job TWICE, not FIVE TIMES (see screens)
            alpanshin Alexandr Panshin made changes -
            Description 1) I created sample job (see "sample_job_config.xml" attachment) which has "PROJECT_NAME" parameter
            2) I want to run it in parallel 5 times with different "PROJECT_NAME" values. I created following script for this:

            {code:java}
            pipeline {
                agent none
                stages {
                    stage('Processing projects') {
                        steps {
                            script {
                                def projectsBuilds = [:]

                                def labelParameters0 = []
                                labelParameters0.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters0.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project0'])
                                projectsBuilds['project0'] = {
                                    stage(String.format('%s execution', 'project0')) {
                                        build job: 'Sample-Job', parameters: labelParameters0
                                    }
                                }

                                def labelParameters1 = []
                                labelParameters0.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters0.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project1'])
                                projectsBuilds['project1'] = {
                                    stage(String.format('%s execution', 'project1')) {
                                        build job: 'Sample-Job', parameters: labelParameters1
                                    }
                                }

                                def labelParameters2 = []
                                labelParameters0.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters0.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project2'])
                                projectsBuilds['project2'] = {
                                    stage(String.format('%s execution', 'project2')) {
                                        build job: 'Sample-Job', parameters: labelParameters2
                                    }
                                }

                                def labelParameters3 = []
                                labelParameters0.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters0.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project3'])
                                projectsBuilds['project3'] = {
                                    stage(String.format('%s execution', 'project3')) {
                                        build job: 'Sample-Job', parameters: labelParameters3
                                    }
                                }

                                def labelParameters4 = []
                                labelParameters0.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters0.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project4'])
                                projectsBuilds['project4'] = {
                                    stage(String.format('%s execution', 'project4')) {
                                        build job: 'Sample-Job', parameters: labelParameters4
                                    }
                                }

                                parallel projectsBuilds;
                            }
                        }
                    }
                }
            }
            {code}

            3) However, when I run this script it only invokes job TWICE, not FIVE TIMES (see screens)
            1) I created sample job (see "sample_job_config.xml" attachment) which has "PROJECT_NAME" parameter
            2) I want to run it in parallel 5 times with different "PROJECT_NAME" values. I created following script for this:

            {code:java}
            pipeline {
                agent none
                stages {
                    stage('Processing projects') {
                        steps {
                            script {
                                def projectsBuilds = [:]

                                def labelParameters = []
                                labelParameters.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project0'])
                                projectsBuilds['project0'] = {
                                    stage(String.format('%s execution', 'project0')) {
                                        build job: 'Sample-Job', parameters: labelParameters
                                    }
                                }

                                labelParameters = []
                                labelParameters.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project1'])
                                projectsBuilds['project1'] = {
                                    stage(String.format('%s execution', 'project1')) {
                                        build job: 'Sample-Job', parameters: labelParameters
                                    }
                                }

                                labelParameters = []
                                labelParameters.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project2'])
                                projectsBuilds['project2'] = {
                                    stage(String.format('%s execution', 'project2')) {
                                        build job: 'Sample-Job', parameters: labelParameters
                                    }
                                }

                                labelParameters = []
                                labelParameters.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project3'])
                                projectsBuilds['project3'] = {
                                    stage(String.format('%s execution', 'project3')) {
                                        build job: 'Sample-Job', parameters: labelParameters
                                    }
                                }

                                labelParameters = []
                                labelParameters.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project4'])
                                projectsBuilds['project4'] = {
                                    stage(String.format('%s execution', 'project4')) {
                                        build job: 'Sample-Job', parameters: labelParameters
                                    }
                                }

                                parallel projectsBuilds;
                            }
                        }
                    }
                }
            }

            {code}

            3) However, when I run this script it only invokes job ONCE, not FIVE TIMES (see screens)
            Summary Parallel execution of same job N times with different parameters leads to only 2 executions of the job Parallel execution of same job N times with different parameters leads to only 1 execution of the job
            alpanshin Alexandr Panshin made changes -
            Attachment screen0.jpg [ 45723 ]
            alpanshin Alexandr Panshin made changes -
            Attachment screen1.jpg [ 45722 ]
            alpanshin Alexandr Panshin made changes -
            Attachment screen.jpg [ 45725 ]
            alpanshin Alexandr Panshin made changes -
            Description 1) I created sample job (see "sample_job_config.xml" attachment) which has "PROJECT_NAME" parameter
            2) I want to run it in parallel 5 times with different "PROJECT_NAME" values. I created following script for this:

            {code:java}
            pipeline {
                agent none
                stages {
                    stage('Processing projects') {
                        steps {
                            script {
                                def projectsBuilds = [:]

                                def labelParameters = []
                                labelParameters.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project0'])
                                projectsBuilds['project0'] = {
                                    stage(String.format('%s execution', 'project0')) {
                                        build job: 'Sample-Job', parameters: labelParameters
                                    }
                                }

                                labelParameters = []
                                labelParameters.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project1'])
                                projectsBuilds['project1'] = {
                                    stage(String.format('%s execution', 'project1')) {
                                        build job: 'Sample-Job', parameters: labelParameters
                                    }
                                }

                                labelParameters = []
                                labelParameters.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project2'])
                                projectsBuilds['project2'] = {
                                    stage(String.format('%s execution', 'project2')) {
                                        build job: 'Sample-Job', parameters: labelParameters
                                    }
                                }

                                labelParameters = []
                                labelParameters.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project3'])
                                projectsBuilds['project3'] = {
                                    stage(String.format('%s execution', 'project3')) {
                                        build job: 'Sample-Job', parameters: labelParameters
                                    }
                                }

                                labelParameters = []
                                labelParameters.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project4'])
                                projectsBuilds['project4'] = {
                                    stage(String.format('%s execution', 'project4')) {
                                        build job: 'Sample-Job', parameters: labelParameters
                                    }
                                }

                                parallel projectsBuilds;
                            }
                        }
                    }
                }
            }

            {code}

            3) However, when I run this script it only invokes job ONCE, not FIVE TIMES (see screens)
            1) I created sample job (see "sample_job_config.xml" attachment) which has "PROJECT_NAME" parameter
             2) I want to run it in parallel 5 times with different "PROJECT_NAME" values. I created following script for this:
            {code:java}
            pipeline {
                agent none
                stages {
                    stage('Processing projects') {
                        steps {
                            script {
                                def projectsBuilds = [:]

                                def labelParameters = []
                                labelParameters.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project0'])
                                projectsBuilds['project0'] = {
                                    stage(String.format('%s execution', 'project0')) {
                                        build job: 'Sample-Job', parameters: labelParameters
                                    }
                                }

                                labelParameters = []
                                labelParameters.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project1'])
                                projectsBuilds['project1'] = {
                                    stage(String.format('%s execution', 'project1')) {
                                        build job: 'Sample-Job', parameters: labelParameters
                                    }
                                }

                                labelParameters = []
                                labelParameters.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project2'])
                                projectsBuilds['project2'] = {
                                    stage(String.format('%s execution', 'project2')) {
                                        build job: 'Sample-Job', parameters: labelParameters
                                    }
                                }

                                labelParameters = []
                                labelParameters.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project3'])
                                projectsBuilds['project3'] = {
                                    stage(String.format('%s execution', 'project3')) {
                                        build job: 'Sample-Job', parameters: labelParameters
                                    }
                                }

                                labelParameters = []
                                labelParameters.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project4'])
                                projectsBuilds['project4'] = {
                                    stage(String.format('%s execution', 'project4')) {
                                        build job: 'Sample-Job', parameters: labelParameters
                                    }
                                }

                                parallel projectsBuilds;
                            }
                        }
                    }
                }
            }

            {code}
            3) However, when I run this script it only invokes job ONCE, not FIVE TIMES (see screen)
            alpanshin Alexandr Panshin made changes -
            Priority Major [ 3 ] Minor [ 4 ]
            alpanshin Alexandr Panshin made changes -
            Description 1) I created sample job (see "sample_job_config.xml" attachment) which has "PROJECT_NAME" parameter
             2) I want to run it in parallel 5 times with different "PROJECT_NAME" values. I created following script for this:
            {code:java}
            pipeline {
                agent none
                stages {
                    stage('Processing projects') {
                        steps {
                            script {
                                def projectsBuilds = [:]

                                def labelParameters = []
                                labelParameters.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project0'])
                                projectsBuilds['project0'] = {
                                    stage(String.format('%s execution', 'project0')) {
                                        build job: 'Sample-Job', parameters: labelParameters
                                    }
                                }

                                labelParameters = []
                                labelParameters.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project1'])
                                projectsBuilds['project1'] = {
                                    stage(String.format('%s execution', 'project1')) {
                                        build job: 'Sample-Job', parameters: labelParameters
                                    }
                                }

                                labelParameters = []
                                labelParameters.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project2'])
                                projectsBuilds['project2'] = {
                                    stage(String.format('%s execution', 'project2')) {
                                        build job: 'Sample-Job', parameters: labelParameters
                                    }
                                }

                                labelParameters = []
                                labelParameters.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project3'])
                                projectsBuilds['project3'] = {
                                    stage(String.format('%s execution', 'project3')) {
                                        build job: 'Sample-Job', parameters: labelParameters
                                    }
                                }

                                labelParameters = []
                                labelParameters.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                labelParameters.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: 'project4'])
                                projectsBuilds['project4'] = {
                                    stage(String.format('%s execution', 'project4')) {
                                        build job: 'Sample-Job', parameters: labelParameters
                                    }
                                }

                                parallel projectsBuilds;
                            }
                        }
                    }
                }
            }

            {code}
            3) However, when I run this script it only invokes job ONCE, not FIVE TIMES (see screen)
            1) I created sample job (see "sample_job_config.xml" attachment) which has "PROJECT_NAME" parameter
             2) I want to run it in parallel 5 times with different "PROJECT_NAME" values. I created following script for this:
            {code:java}
            pipeline {
                agent none
                stages {
                    stage('Processing projects') {
                        steps {
                            script {
                                def projects = ['project0', 'project1', 'project2', 'project3', 'project4']
                                def projectsBuilds = [:]

                                def labelParameters
                                for (project in projects) {
                                    //JENKINS-55426 need to save iterated variable value into intermediate variable, otherwise there will be last value from collection for all jobs
                                    def currentProject = project
                                    labelParameters = []
                                    labelParameters.add([$class: 'LabelParameterValue', name: 'node', label: 'linux'])
                                    labelParameters.add([$class: "StringParameterValue", name: "PROJECT_NAME", value: currentProject])

                                    projectsBuilds[currentProject] = {
                                        stage(String.format('%s execution', currentProject)) {
                                            build job: 'Sample-Job', parameters: labelParameters
                                        }
                                    }
                                }

                                parallel projectsBuilds;
                            }
                        }
                    }
                }
            }
            {code}
            3) However, when I run this script it only invokes job ONCE, not FIVE TIMES (see screen)
            alpanshin Alexandr Panshin made changes -
            Link This issue blocks JENKINS-55748 [ JENKINS-55748 ]
            alpanshin Alexandr Panshin made changes -
            Resolution Duplicate [ 3 ]
            Status Open [ 1 ] Closed [ 6 ]

              People

              Assignee:
              vjuranek vjuranek
              Reporter:
              alpanshin Alexandr Panshin
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: