Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-60213

P4_CLIENT environment variable non-reentrant

XMLWordPrintable

      Version 1.10.6 of the p4-plugin causes issues when multiple jobs are ran in parallel.

      The result is that the p4 created environment variables of one job can corrupt those in another job that is running at the same time.

      This bug wasn't in 1.10.5

      Steps to reproduce issue:

      • Create a new Jenkins test job with the following settings
        • General:
          • Execute concurrent builds if necessary
          • Use custom workspace
            • Directory 
              /tmp/${NODE_NAME}-${JOB_NAME}-${EXECUTOR_NUMBER}
        • Source Code Management
          • Perforce Software:
            • Workspace behaviour: Template (view generated for each node)
              • Template workspace: Your test template
              • Workspace Name Format being 
                 jenkins-${NODE_NAME}-${JOB_NAME}-${EXECUTOR_NUMBER}
      • Add a build step

      To see the issue:

      • Kick off two runs of the newly creates client

      Result:

      • Pre 1.10.5 both would have worked and both will set the expected P4 environment variables.
      • 1.10.6 gets confused. So in my case:

        Job1 Environment variables

        P4_CLIENT	_jenkins-master-ClientClashTest-0
WORKSPACE	/tmp/master-ClientClashTest-0

        Job2 Environment variables (THIS IS THE ISSUE – Wrong P4_CLIENT value)

        P4_CLIENT	_jenkins-master-ClientClashTest-0
WORKSPACE	/tmp/master-ClientClashTest-1

        —

        If I reinstall the 1.10.5 (or earlier) plugin then for job 2 I would see:

        P4_CLIENT	_jenkins-master-ClientClashTest-1
WORKSPACE	/tmp/master-ClientClashTest-1

        1. JENKINS-60213__JobConfig.png
          JENKINS-60213__JobConfig.png
          254 kB
        2. screenshot-1.png
          screenshot-1.png
          103 kB
        3. Build History.png
          Build History.png
          9 kB

          [JENKINS-60213] P4_CLIENT environment variable non-reentrant

          James Bateman created issue -
          James Bateman made changes -
          Priority Original: Minor [ 4 ] New: Major [ 3 ]

          Karl Wirth added a comment -

          Hi jbateman - Thanks for letting us know.

          This looks like a freestyle job. Is that correct? Please also attach the code you are using to display the variables.

          Thanks in advance,

          Karl

          Karl Wirth added a comment - Hi jbateman - Thanks for letting us know. This looks like a freestyle job. Is that correct? Please also attach the code you are using to display the variables. Thanks in advance, Karl
          James Bateman made changes -
          Attachment New: Build History.png [ 49551 ]
          James Bateman made changes -
          Attachment New: screenshot-1.png [ 49552 ]
          James Bateman made changes -
          Attachment New: JENKINS-60213__JobConfig.png [ 49553 ]

          James Bateman added a comment - - edited

          More info.

          • We are running Jenkins 2.190.2 on our test Jenkins server
          • We have just upgraded all plugins to the latest versions
          • We have tested every verison of the P4 plugin from 1.9.3 --> 1.10.6 (Only 1.10.6 has this particular issue)
          • To see this particular bug, kick off two jobs so they are running in parallel
          • Console output of the two jobs (in this case #32 + #33) – You will see that the 'P4_CLIENT' environment variable for job #32 is wrong
          • Job configuration

          NOTE: If you have the InjectedEnvVars plugin installed then you can also inspect the environment variables of a previously ran job and again this will show the same issue. To be able to see the P4 variables you will need to add a 'Inject environment variables' build step (it can be blank). Looking at the environment variables both P4_CLIENT and P4_ROOT can become corrupted.

          James Bateman added a comment - - edited More info. We are running Jenkins 2.190.2 on our test Jenkins server We have just upgraded all plugins to the latest versions We have tested every verison of the P4 plugin from 1.9.3 --> 1.10.6 (Only 1.10.6 has this particular issue) To see this particular bug, kick off two jobs so they are running in parallel Console output of the two jobs (in this case #32 + #33) – You will see that the 'P4_CLIENT' environment variable for job #32 is wrong Job configuration NOTE: If you have the InjectedEnvVars plugin installed then you can also inspect the environment variables of a previously ran job and again this will show the same issue. To be able to see the P4 variables you will need to add a ' Inject environment variables ' build step (it can be blank). Looking at the environment variables both P4_CLIENT  and P4_ROOT can become corrupted.

          Karl Wirth added a comment -

          Hi jbateman - That's great thanks. I'll test here and get back to you.

          Regards,

          Karl

          Karl Wirth added a comment - Hi jbateman - That's great thanks. I'll test here and get back to you. Regards, Karl

          Karl Wirth added a comment -

          Hi jbateman - Thanks again. I was easily able to reproduce this by spamming the 'Build Now' button 3 times on 1.10.6 and the following script:

          #!/bin/bash
echo "=========================================="
echo "WORKSPACE:            ${WORKSPACE}"
echo "P4_CLIENT:            ${P4_CLIENT}"
echo "EXECUTOR NUMBER:      ${EXECUTOR_NUMBER}"
echo "=========================================="
sleep 30

          1.10.6 behavior uses same workspace:

           

          [JENKINS-60213-ConcurrentVariables@2] $ /bin/bash /tmp/jenkins7949726486877689022.sh
==========================================
WORKSPACE:            /var/lib/jenkins/workspace/JENKINS-60213-ConcurrentVariables@2
P4_CLIENT:            jenkins-master-JENKINS-60213-ConcurrentVariables-3
EXECUTOR NUMBER:     2
==========================================

           

           

           

          [JENKINS-60213-ConcurrentVariables@3] $ /bin/bash /tmp/jenkins8997855946897277031.sh
==========================================
WORKSPACE:            /var/lib/jenkins/workspace/JENKINS-60213-ConcurrentVariables@3
P4_CLIENT:            jenkins-master-JENKINS-60213-ConcurrentVariables-3
EXECUTOR NUMBER:     6
==========================================

           

           

          On 1.10.4 I see different workspaces:

          Build 17:
[JENKINS-60213-ConcurrentVariables] $ /bin/bash /tmp/jenkins5428347631299328773.sh
==========================================
WORKSPACE:            /var/lib/jenkins/workspace/JENKINS-60213-ConcurrentVariables
P4_CLIENT:            jenkins-master-JENKINS-60213-ConcurrentVariables-3
EXECUTOR NUMBER:     3
==========================================

Build 18:
[JENKINS-60213-ConcurrentVariables@2] $ /bin/bash /tmp/jenkins7094128855131134804.sh
==========================================
WORKSPACE:            /var/lib/jenkins/workspace/JENKINS-60213-ConcurrentVariables@2
P4_CLIENT:            jenkins-master-JENKINS-60213-ConcurrentVariables-2
EXECUTOR NUMBER:     2
==========================================

Build 19:
[JENKINS-60213-ConcurrentVariables@3] $ /bin/bash /tmp/jenkins387246113188966258.sh
==========================================
WORKSPACE:            /var/lib/jenkins/workspace/JENKINS-60213-ConcurrentVariables@3
P4_CLIENT:            jenkins-master-JENKINS-60213-ConcurrentVariables-0
EXECUTOR NUMBER:     0
==========================================

          On 1.10.6 I see different workspaces sometimes.

           

          Test 1 works as expected:

          Build 23:
[JENKINS-60213-ConcurrentVariables] $ /bin/bash /tmp/jenkins6724175724105556673.sh
==========================================
WORKSPACE:            /var/lib/jenkins/workspace/JENKINS-60213-ConcurrentVariables
P4_CLIENT:            jenkins-master-JENKINS-60213-ConcurrentVariables-2
EXECUTOR NUMBER:     2
==========================================

Build 24:[JENKINS-60213-ConcurrentVariables@2] $ /bin/bash /tmp/jenkins2594984598801278037.sh
==========================================
WORKSPACE:            /var/lib/jenkins/workspace/JENKINS-60213-ConcurrentVariables@2
P4_CLIENT:            jenkins-master-JENKINS-60213-ConcurrentVariables-9
EXECUTOR NUMBER:     9
==========================================

Build 25:[JENKINS-60213-ConcurrentVariables@3] $ /bin/bash /tmp/jenkins7864300692310335405.sh
==========================================
WORKSPACE:            /var/lib/jenkins/workspace/JENKINS-60213-ConcurrentVariables@3
P4_CLIENT:            jenkins-master-JENKINS-60213-ConcurrentVariables-7
EXECUTOR NUMBER:     7
==========================================

          Test 2 uses the same P4_CLIENT on 2 out of 3 concurrent executors:

          Build 26:
[JENKINS-60213-ConcurrentVariables] $ /bin/bash /tmp/jenkins7858524505176489757.sh
==========================================
WORKSPACE:            /var/lib/jenkins/workspace/JENKINS-60213-ConcurrentVariables
P4_CLIENT:            jenkins-master-JENKINS-60213-ConcurrentVariables-6
EXECUTOR NUMBER:     8
==========================================

Build 27:[JENKINS-60213-ConcurrentVariables@2] $ /bin/bash /tmp/jenkins2825147022357472107.sh
==========================================
WORKSPACE:            /var/lib/jenkins/workspace/JENKINS-60213-ConcurrentVariables@2
P4_CLIENT:            jenkins-master-JENKINS-60213-ConcurrentVariables-6
EXECUTOR NUMBER:     6
==========================================

Build 28:[JENKINS-60213-ConcurrentVariables@3] $ /bin/bash /tmp/jenkins8052525907828215068.sh
==========================================
WORKSPACE:            /var/lib/jenkins/workspace/JENKINS-60213-ConcurrentVariables@3
P4_CLIENT:            jenkins-master-JENKINS-60213-ConcurrentVariables-4
EXECUTOR NUMBER:     4
==========================================

          IMPORTANT NOTE: The P4CLIENT used in the job for build 8 is 'jenkins-master-JENKINS-60213-ConcurrentVariables-8'. Therefore it correctly uses a different client but sets the env variable to the wrong value.

           

          Passing to developers.

           

           

           

          Karl Wirth added a comment - Hi jbateman - Thanks again. I was easily able to reproduce this by spamming the 'Build Now' button 3 times on 1.10.6 and the following script: #!/bin/bash echo "==========================================" echo "WORKSPACE: ${WORKSPACE}" echo "P4_CLIENT: ${P4_CLIENT}" echo "EXECUTOR NUMBER: ${EXECUTOR_NUMBER}" echo "==========================================" sleep 30 1.10.6 behavior uses same workspace:   [JENKINS-60213-ConcurrentVariables@2] $ /bin/bash /tmp/jenkins7949726486877689022.sh ========================================== WORKSPACE: / var /lib/jenkins/workspace/JENKINS-60213-ConcurrentVariables@2 P4_CLIENT: jenkins-master-JENKINS-60213-ConcurrentVariables-3 EXECUTOR NUMBER: 2 ==========================================       [JENKINS-60213-ConcurrentVariables@3] $ /bin/bash /tmp/jenkins8997855946897277031.sh ========================================== WORKSPACE: / var /lib/jenkins/workspace/JENKINS-60213-ConcurrentVariables@3 P4_CLIENT: jenkins-master-JENKINS-60213-ConcurrentVariables-3 EXECUTOR NUMBER: 6 ==========================================     On 1.10.4 I see different workspaces: Build 17: [JENKINS-60213-ConcurrentVariables] $ /bin/bash /tmp/jenkins5428347631299328773.sh ========================================== WORKSPACE: / var /lib/jenkins/workspace/JENKINS-60213-ConcurrentVariables P4_CLIENT: jenkins-master-JENKINS-60213-ConcurrentVariables-3 EXECUTOR NUMBER: 3 ========================================== Build 18: [JENKINS-60213-ConcurrentVariables@2] $ /bin/bash /tmp/jenkins7094128855131134804.sh ========================================== WORKSPACE: / var /lib/jenkins/workspace/JENKINS-60213-ConcurrentVariables@2 P4_CLIENT: jenkins-master-JENKINS-60213-ConcurrentVariables-2 EXECUTOR NUMBER: 2 ========================================== Build 19: [JENKINS-60213-ConcurrentVariables@3] $ /bin/bash /tmp/jenkins387246113188966258.sh ========================================== WORKSPACE: / var /lib/jenkins/workspace/JENKINS-60213-ConcurrentVariables@3 P4_CLIENT: jenkins-master-JENKINS-60213-ConcurrentVariables-0 EXECUTOR NUMBER: 0 ========================================== On 1.10.6 I see different workspaces sometimes.   Test 1 works as expected: Build 23: [JENKINS-60213-ConcurrentVariables] $ /bin/bash /tmp/jenkins6724175724105556673.sh ========================================== WORKSPACE: / var /lib/jenkins/workspace/JENKINS-60213-ConcurrentVariables P4_CLIENT: jenkins-master-JENKINS-60213-ConcurrentVariables-2 EXECUTOR NUMBER: 2 ========================================== Build 24:[JENKINS-60213-ConcurrentVariables@2] $ /bin/bash /tmp/jenkins2594984598801278037.sh ========================================== WORKSPACE: / var /lib/jenkins/workspace/JENKINS-60213-ConcurrentVariables@2 P4_CLIENT: jenkins-master-JENKINS-60213-ConcurrentVariables-9 EXECUTOR NUMBER: 9 ========================================== Build 25:[JENKINS-60213-ConcurrentVariables@3] $ /bin/bash /tmp/jenkins7864300692310335405.sh ========================================== WORKSPACE: / var /lib/jenkins/workspace/JENKINS-60213-ConcurrentVariables@3 P4_CLIENT: jenkins-master-JENKINS-60213-ConcurrentVariables-7 EXECUTOR NUMBER: 7 ========================================== Test 2 uses the same P4_CLIENT on 2 out of 3 concurrent executors: Build 26: [JENKINS-60213-ConcurrentVariables] $ /bin/bash /tmp/jenkins7858524505176489757.sh ========================================== WORKSPACE: / var /lib/jenkins/workspace/JENKINS-60213-ConcurrentVariables P4_CLIENT: jenkins-master-JENKINS-60213-ConcurrentVariables-6 EXECUTOR NUMBER: 8 ========================================== Build 27:[JENKINS-60213-ConcurrentVariables@2] $ /bin/bash /tmp/jenkins2825147022357472107.sh ========================================== WORKSPACE: / var /lib/jenkins/workspace/JENKINS-60213-ConcurrentVariables@2 P4_CLIENT: jenkins-master-JENKINS-60213-ConcurrentVariables-6 EXECUTOR NUMBER: 6 ========================================== Build 28:[JENKINS-60213-ConcurrentVariables@3] $ /bin/bash /tmp/jenkins8052525907828215068.sh ========================================== WORKSPACE: / var /lib/jenkins/workspace/JENKINS-60213-ConcurrentVariables@3 P4_CLIENT: jenkins-master-JENKINS-60213-ConcurrentVariables-4 EXECUTOR NUMBER: 4 ========================================== IMPORTANT NOTE: The P4CLIENT used in the job for build 8 is 'jenkins-master- JENKINS-60213 -ConcurrentVariables-8'. Therefore it correctly uses a different client but sets the env variable to the wrong value.   Passing to developers.      
          Karl Wirth made changes -
          Labels New: P4_VERIFY

            msmeeth Matthew Smeeth
            jbateman James Bateman
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: