Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-55426

Using "for in" loop for generating tasks for "parallel" execution, causes all tasks to have last value from iterated collection

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Minor Minor
    • groovy-plugin
    • Jenkins 2.150.1, Pipeline-Groovy Plugin 2.57

      When I try to generate tasks for parallel execution using the following code:

      import jenkins.model.Jenkins
      
      pipeline {
          agent none
          stages {
              stage('Preparing machines') {
                  steps {
                      script {
                          def machines = ['agent-windows0', 'agent-windows1', 'agent-redhat0', 'agent-redhat1']
                          def machinePreparations = [:]
      
                          for (machine in machines) {
                              def labelParameters = []
                              labelParameters.add([$class: 'NodeParameterValue', name: 'node_name', labels: [machine], nodeEligibility: [$class: 'AllNodeEligibility']])
                              labelParameters.add([$class: 'StringParameterValue', name: 'ARBITRARY_PARAMETER', value: 'ARBITRARY_VALUE'])
      
                              machinePreparations[machine] = {
                                  stage(machine + ' preparation') {
                                      build job: '../Preparation/' + machine, parameters: labelParameters
      
                                      def node = Jenkins.get().nodes.find({it.name == machine})
                                      node.setLabelString(node.getLabelString() + 'successful')
                                      println(String.format("Job with machine %s successfully done!", machine))
                                  }
                              }
                          }
      
                          parallel machinePreparations
                      }
                  }
              }
          }
      }
      

      I have all the tasks receiving last value from collection ("agent-redhat1") - see screen attached.
      This can be partially fixed by saving "machine" variable into another intermediate variable and using this intermediate variable instead:

      import jenkins.model.Jenkins
      
      pipeline {
          agent none
          stages {
              stage('Preparing machines') {
                  steps {
                      script {
                          def machines = ['agent-windows0', 'agent-windows1', 'agent-redhat0', 'agent-redhat1']
                          def machinePreparations = [:]
      
                          for (machine in machines) {
                              def agentName = machine
                              def labelParameters = []
                              labelParameters.add([$class: 'NodeParameterValue', name: 'node_name', labels: [agentName], nodeEligibility: [$class: 'AllNodeEligibility']])
                              labelParameters.add([$class: 'StringParameterValue', name: 'ARBITRARY_PARAMETER', value: 'ARBITRARY_VALUE'])
      
                              machinePreparations[agentName] = {
                                  stage(agentName + ' preparation') {
                                      build job: '../Preparation/' + agentName, parameters: labelParameters
      
                                      def node = Jenkins.get().nodes.find({it.name == agentName})
                                      node.setLabelString(node.getLabelString() + 'successful')
                                      println(String.format("Job with machine %s successfully done!", agentName))
                                  }
                              }
                          }
      
                          parallel machinePreparations
                      }
                  }
              }
          }
      }
      

      However, this also not fixing problem entirely, see JENKINS-55425

          [JENKINS-55426] Using "for in" loop for generating tasks for "parallel" execution, causes all tasks to have last value from iterated collection

          Stuart Rowe added a comment -

          Avoid using "for-in" loops. Instead try:

          machines.each { machine ->
              def agentName = machine
              def labelParameters = []
              labelParameters.add([$class: 'NodeParameterValue', name: 'node_name', labels: [agentName], nodeEligibility: [$class: 'AllNodeEligibility']])
              labelParameters.add([$class: 'StringParameterValue', name: 'ARBITRARY_PARAMETER', value: 'ARBITRARY_VALUE'])
          
              machinePreparations[agentName] = {
                  stage(agentName + ' preparation') {
                      build job: '../Preparation/' + agentName, parameters: labelParameters
          
                      def node = Jenkins.get().nodes.find({it.name == agentName})
                      node.setLabelString(node.getLabelString() + 'successful')
                      println(String.format("Job with machine %s successfully done!", agentName))
                  }
              }
          }

           

           

           

          Stuart Rowe added a comment - Avoid using "for-in" loops. Instead try: machines.each { machine -> def agentName = machine def labelParameters = [] labelParameters.add([$class: 'NodeParameterValue' , name: 'node_name' , labels: [agentName], nodeEligibility: [$class: 'AllNodeEligibility' ]]) labelParameters.add([$class: 'StringParameterValue' , name: 'ARBITRARY_PARAMETER' , value: 'ARBITRARY_VALUE' ]) machinePreparations[agentName] = { stage(agentName + ' preparation' ) { build job: '../Preparation/' + agentName, parameters: labelParameters def node = Jenkins.get().nodes.find({it.name == agentName}) node.setLabelString(node.getLabelString() + 'successful' ) println( String .format( "Job with machine %s successfully done!" , agentName)) } } }      

          Sergei S added a comment - - edited

          I catched similar bug with ".each" loop inside parallel stages.

          Jenkins 2.222(tested on 2.232 too), pipeline groovy plugin 2.80 

          def deploymentStages = [:]
          configGlobal = ["stage-a":["childMap":["key1":"val-1-stage-A", "key2":"val-2-stage-A", "childList":["child-list-value-stage-A"]]], 
                          "stage-b":["childMap":["key1":"val-1-stage-B", "key2":"val-2-stage-B", "childList":["child-list-value-stage-B"]]]]
          
              pipeline {
                  agent any
          
                  stages {
                      stage('Parent') {
                          steps {
                              script {
                                  configGlobal.each { item -> 
                                      deploymentStages["${item.key}"] = {
                                          stage("${item.key}") {
                                              config = item.value.childMap
                                              newList = [config.key1, config.key2]
                                              echo "before loop"
                                              echo newList.toString()
                                              config.childList.each                { value ->      
                                                  newList.add(value)
                                              }
                                              echo "after loop"
                                              echo newList.toString()
                                          }
                                      }
                                  }
                                  parallel deploymentStages
                              }
                          }
                      }
                  }
                  
              }
          

           Output:

          Sergei S added a comment - - edited I catched similar bug with ".each" loop inside parallel stages. Jenkins 2.222(tested on 2.232 too), pipeline groovy plugin 2.80  def deploymentStages = [:] configGlobal = [ "stage-a" :[ "childMap" :[ "key1" : "val-1-stage-A" , "key2" : "val-2-stage-A" , "childList" :[ "child-list-value-stage-A" ]]], "stage-b" :[ "childMap" :[ "key1" : "val-1-stage-B" , "key2" : "val-2-stage-B" , "childList" :[ "child-list-value-stage-B" ]]]] pipeline { agent any stages { stage( 'Parent' ) { steps { script { configGlobal.each { item -> deploymentStages[ "${item.key}" ] = { stage( "${item.key}" ) { config = item.value.childMap newList = [config.key1, config.key2] echo "before loop" echo newList.toString() config.childList.each { value -> newList.add(value) } echo "after loop" echo newList.toString() } } } parallel deploymentStages } } } } }  Output:

          Volmar added a comment -

          I was running thru the same issue and could get it solved with the second closure

              def tasks=[:]
              list.each { item ->
                tasks["${item}"] = { ->
                   build job: "${item}", parameters: [
                        [$class: 'StringParameterValue', name: 'branch', value: "${params.branch}"]
                      ], propagate: false
                  }
              }
          

          Volmar added a comment - I was running thru the same issue and could get it solved with the second closure def tasks=[:] list.each { item -> tasks[ "${item}" ] = { -> build job: "${item}" , parameters: [ [$class: 'StringParameterValue' , name: 'branch' , value: "${params.branch}" ] ], propagate: false } }

          Mark added a comment -

          If you're having trouble with for loops in this scenario, it's possible simply adding def to a variable definition will resolve the issue.

          Mark added a comment - If you're having trouble with for loops in this scenario, it's possible simply adding def to a variable definition will resolve the issue.

            vjuranek vjuranek
            alpanshin Alexandr Panshin
            Votes:
            2 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated: