Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-55040

Problem sharing data objects between parallel build stages

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Major Major
    • pipeline
    • None
    • Jenkins v2.148
      Pipeline v2.6

      I recently found a bug which causes local data created in two closures run by a parallel build operation in Jenkins Pipeline. It seems that variables within closures somehow share memory space or something with one another, causing conflicts when the code is run in parallel. Here is some example code that should exploit the problem I hit:

       
      parallel ( "thread1": {
      node()

      { def a = git(branch: 'BranchA', credentialsId: 'MyCreds', url: 'url/to/git/repo.git') def b = "1" def c = sh(returnStdout: true, script: 'echo 1').trim() echo "git commit " + a.GIT_COMMIT echo "b is " + b echo "c is " + c }

      }, "thread2": {
      node()

      { def a = git(branch: 'BranchB', credentialsId: 'MyCreds', url: 'url/to/git/repo.git') def b = "2" def c = sh(returnStdout: true, script: 'echo 2').trim() echo "git commit " + a.GIT_COMMIT echo "b is " + b echo "c is " + c }

      }
      )
      In this example, I would expect the output to be something like

      [thread1] git commit <BranchA_Hash>
      [thread1] b is 1
      [thread1] c is 1
      [thread2] git commit <BranchB_Hash>
      [thread2] b is 2
      [thread2] c is 2

      What I actually get is this:

      [thread1] git commit <BranchA_Hash>
      [thread1] b is 1
      [thread1] c is 1
      [thread2] git commit <BranchA_Hash>
      [thread2] b is 2
      [thread2] c is 2

      While the 'b' and 'c' variables seem to get unique values for each of the parallel threads, the 'a' value becomes ambiguous, always returning a single Git commit hash.

      Further, I've confirmed that which Git commit hash that is shown represents the one that is executed "first" in the parallel stages. So, for example, if you put a small "sleep 5" at the top of thread1, then you get the Git commit has from BranchB in both echo statements, and if you move the sleep statement to the top of thread2 you get the has from BranchA.

      Also interesting is that although the 'a' variable seems to get confused based on the thread execution, the other variables do not. I thought at first that maybe assigning a static value to the variable may have explained the discrepancy like in the case of the 'b' variable so I added a dynamically executed shell step to the mix for variable 'c' and I get the same results as b... so I'm not sure how / why / what is going on here.

            Unassigned Unassigned
            leedega Kevin Phillips
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: