Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-64707

[P4 Plugin] P4_ROOT is not set properly - pointing to build workspace instead of p4 client root

      The following problem was observed in a pipeline type job:

      After the perforce scm step is executed the P4_ROOT environment variable is not set properly. It is pointing to the build workspace instead of the p4 client root of the client used during the scm step. I have tested it using static workspace behavior and the p4 preview option. Basically the P4_ROOT is the same as WORKSPACE environment variable, but it should be set to p4 clients root.

       

      agent{
        node{
            label "node1"
            customWorkspace "C:\\Custom\\BuildWorkspacePath"
        }
      }
      stages{
        stage('Test'){
          steps{
            // Perforce client myP4Client has p4 root set to C:\\myP4Client
            checkout perforce(credential: 'myP4Creds', populate: previewOnly(quiet: true), workspace: staticSpec(charset: 'none', name: 'myP4Client', pinHost: false))
            // P4_ROOT will be set to "C:\\Custom\\BuildWorkspacePath" instead of "C:\\myP4Client"
            echo "${currentBuild.rawBuild.getEnvironment()}"
          }
        }
      }
      
      

          [JENKINS-64707] [P4 Plugin] P4_ROOT is not set properly - pointing to build workspace instead of p4 client root

          Karl Wirth added a comment -

          Hi plawniczuk . At the moment you cannot set a workspace root as you have found because the build needs to be done under the Jenkins workspace root. In the past I have recomended using the 'ws' directive. For example I give an example here:

           

               https://issues.jenkins.io/browse/JENKINS-47005

           

          Is 'customWorkspace' different to 'ws'? Can you please try 'ws' and see if that solves the problem. Also note that if you are using multiple build slaves and the build can run on any slave please ensure that the node name and job appear in the client name of any real scripts. For example:

              jenkins-${NODE_NAME}${JOB_NAME}${EXECUTOR_NUMBER}

           

           

          Karl Wirth added a comment - Hi plawniczuk . At the moment you cannot set a workspace root as you have found because the build needs to be done under the Jenkins workspace root. In the past I have recomended using the 'ws' directive. For example I give an example here:        https://issues.jenkins.io/browse/JENKINS-47005   Is 'customWorkspace' different to 'ws'? Can you please try 'ws' and see if that solves the problem. Also note that if you are using multiple build slaves and the build can run on any slave please ensure that the node name and job appear in the client name of any real scripts. For example:     jenkins-${NODE_NAME} ${JOB_NAME} ${EXECUTOR_NUMBER}    

          Hi Karl,

          'customWorkspace' and 'ws' is basically the same, it will allocate the workspace with given path and the result will be the same as I described it in my ticket: P4_ROOT will point to the value of WORKSPACE env var. 

          I think I did not explain my scenario good enough and the use case in which this behavior is problematic. In our pipelines we use mostly static workspace's and preview as an operation. The rest of p4 operations like: sync, unshelve, revert etc. is done by the bat steps and p4 cli tool, as we would like to have total control of all perforce related operations. We use P4 Plugin mostly to read the head of the client and for all the Jenkins changelog related features. We now have a scenario where workspaces are located inside the folder junctions and p4 operations fails due to wrong pwd (Symbolic Links and Workspace Roots), so getting the real p4 client root with use of plugin in P4_ROOT env var would be really helpful here.

          It means also also that description for P4_ROOT is very misleading (from P4 Plugin documentation):

          P4_ROOT - The Perforce client workspace root path.
          

          It is not perforce client workspace root path, it is the Jenkins build workspace root path.

          Piotr Ławniczuk added a comment - Hi Karl, 'customWorkspace' and 'ws' is basically the same, it will allocate the workspace with given path and the result will be the same as I described it in my ticket: P4_ROOT will point to the value of WORKSPACE env var.  I think I did not explain my scenario good enough and the use case in which this behavior is problematic. In our pipelines we use mostly static workspace's and preview as an operation. The rest of p4 operations like: sync, unshelve, revert etc. is done by the bat steps and p4 cli tool, as we would like to have total control of all perforce related operations. We use P4 Plugin mostly to read the head of the client and for all the Jenkins changelog related features. We now have a scenario where workspaces are located inside the folder junctions and p4 operations fails due to wrong pwd ( Symbolic Links and Workspace Roots ), so getting the real p4 client root with use of plugin in P4_ROOT env var would be really helpful here. It means also also that description for P4_ROOT is very misleading (from P4 Plugin documentation): P4_ROOT - The Perforce client workspace root path. It is not perforce client workspace root path, it is the Jenkins build workspace root path.

          Karl Wirth added a comment -

          Hi plawniczuk

          Thanks. As you have found any workspace used by P4Jenkins has its workspace root overwritten by the Jenkins workspace root. We have the open enhancement request JENKINS-47005 to change this behavior.

           

          However using 'ws' or 'customWorkspace' (thanks for letting me know about that one) should allow you to workaround this. For example here is my Jenkinsfile:

          pipeline {
          agent{
            node{
                label "Windows10"
                customWorkspace "E:\\filestore\\Jenkins\\outside_Windows10Slave22"
            }
          }
          //options { skipDefaultCheckout() }
          stages{
            stage('Test'){
              steps{
                checkout perforce(credential: 'JenkinsMaster', populate: previewOnly(quiet: true), workspace: staticSpec(charset: 'none', name: 'JENKINS-64707-ws', pinHost: false))
                echo "${currentBuild.rawBuild.getEnvironment()}"
                bat "echo p4 -p %P4_PORT% -u %P4_USER% -P %P4_TICKET% -c %P4_CLIENT% client -o"
                bat "p4 -p %P4_PORT% -u %P4_USER% -P %P4_TICKET% -c %P4_CLIENT% client -o"
              }
            }
          }
          
          

          This causes the code to be synced to the path "E:\filestore\Jenkins\outside_Windows10Slave22". For example in the console log:

          12:17:34  P4 Task: syncing files at change: 215349
          12:17:34  (p4):cmd:... p4 sync -p -q E:\filestore\Jenkins\outside_Windows10Slave22/...@215349
          12:17:34  ... totalFileSize 1041
          12:17:34  ... totalFileCount 7
          

          and:

          12:17:41  [BUILD_DISPLAY_NAME:#16, BUILD_ID:16, BUILD_NUMBER:16, BUILD_TAG:jenkins-JENKINS-64707_ws_root-16, BUILD_URL:http://vm-kwirth-swarm182-xenial:8080/job/JENKINS-64707_ws_root/16/, CLASSPATH:, HUDSON_CHANGELOG_FILE:/var/lib/jenkins/jobs/JENKINS-64707_ws_root/builds/16/changelog1.xml, HUDSON_HOME:/var/lib/jenkins, HUDSON_SERVER_COOKIE:c131d783e95d951e, HUDSON_URL:http://vm-kwirth-swarm182-xenial:8080/, JENKINS_HOME:/var/lib/jenkins, JENKINS_SERVER_COOKIE:c131d783e95d951e, JENKINS_URL:http://vm-kwirth-swarm182-xenial:8080/, JOB_BASE_NAME:JENKINS-64707_ws_root, JOB_DISPLAY_URL:http://vm-kwirth-swarm182-xenial:8080/job/JENKINS-64707_ws_root/display/redirect, JOB_NAME:JENKINS-64707_ws_root, JOB_URL:http://vm-kwirth-swarm182-xenial:8080/job/JENKINS-64707_ws_root/, P4_CHANGELIST:215349, P4_CLIENT:JENKINS-64707-ws, P4_PORT:vm-kwirth-swarm182-xenial:1666, P4_ROOT:E:\filestore\Jenkins\outside_Windows10Slave22, P4_TICKET:9058B5D7DC53D417EDFEC9E63AB39BBB, P4_USER:super, P4CLIENT:JENKINS-64707-ws, RUN_CHANGES_DISPLAY_URL:http://vm-kwirth-swarm182-xenial:8080/job/JENKINS-64707_ws_root/16/display/redirect?page=changes, RUN_DISPLAY_URL:http://vm-kwirth-swarm182-xenial:8080/job/JENKINS-64707_ws_root/16/display/redirect]
          

          Also note that originally my workspace root was 'E:\filestore\test' but it has now been changed by P4Jenkins to match the value of 'customWorkspace':

          12:17:42  E:\filestore\Jenkins\outside_Windows10Slave22>p4 -p vm-kwirth-swarm182-xenial:1666 -u super -P 9058B5D7DC53D417EDFEC9E63AB39BBB -c jenkins-Windows10-JENKINS-64707_ws_root-0-thescript client -o 
          12:17:43  # A Perforce Client Specification.
          12:17:43  #
          ...cut...
          12:17:43  
          12:17:43  Client:	jenkins-Windows10-JENKINS-64707_ws_root-0-thescript
          12:17:43  
          12:17:43  Update:	2021/01/27 11:59:06
          12:17:43  
          12:17:43  Access:	2021/01/27 12:17:30
          12:17:43  
          12:17:43  Owner:	super
          12:17:43  
          12:17:43  Root:	E:\filestore\Jenkins\outside_Windows10Slave22   *** THIS HAS BEEN OVERWRITTEN ***
          12:17:43  
          12:17:43  Options:	noallwrite clobber nocompress unlocked nomodtime normdir
          12:17:43  
          12:17:43  SubmitOptions:	submitunchanged
          12:17:43  
          12:17:43  LineEnd:	local
          12:17:43  
          12:17:43  View:
          12:17:43  	//depot/JENKINS-64707_ws_root/... //jenkins-Windows10-JENKINS-64707_ws_root-0-thescript/...
          

           

           

           

           

          Karl Wirth added a comment - Hi plawniczuk Thanks. As you have found any workspace used by P4Jenkins has its workspace root overwritten by the Jenkins workspace root. We have the open enhancement request JENKINS-47005 to change this behavior.   However using 'ws' or 'customWorkspace' (thanks for letting me know about that one) should allow you to workaround this. For example here is my Jenkinsfile: pipeline { agent{ node{ label "Windows10" customWorkspace "E:\\filestore\\Jenkins\\outside_Windows10Slave22" } } //options { skipDefaultCheckout() } stages{ stage( 'Test' ){ steps{ checkout perforce(credential: 'JenkinsMaster' , populate: previewOnly(quiet: true ), workspace: staticSpec(charset: 'none' , name: 'JENKINS-64707-ws' , pinHost: false )) echo "${currentBuild.rawBuild.getEnvironment()}" bat "echo p4 -p %P4_PORT% -u %P4_USER% -P %P4_TICKET% -c %P4_CLIENT% client -o" bat "p4 -p %P4_PORT% -u %P4_USER% -P %P4_TICKET% -c %P4_CLIENT% client -o" } } } This causes the code to be synced to the path "E:\filestore\Jenkins\outside_Windows10Slave22". For example in the console log: 12:17:34 P4 Task: syncing files at change: 215349 12:17:34 (p4):cmd:... p4 sync -p -q E:\filestore\Jenkins\outside_Windows10Slave22/...@215349 12:17:34 ... totalFileSize 1041 12:17:34 ... totalFileCount 7 and: 12:17:41 [BUILD_DISPLAY_NAME:#16, BUILD_ID:16, BUILD_NUMBER:16, BUILD_TAG:jenkins-JENKINS-64707_ws_root-16, BUILD_URL:http: //vm-kwirth-swarm182-xenial:8080/job/JENKINS-64707_ws_root/16/, CLASSPATH:, HUDSON_CHANGELOG_FILE:/ var /lib/jenkins/jobs/JENKINS-64707_ws_root/builds/16/changelog1.xml, HUDSON_HOME:/ var /lib/jenkins, HUDSON_SERVER_COOKIE:c131d783e95d951e, HUDSON_URL:http://vm-kwirth-swarm182-xenial:8080/, JENKINS_HOME:/ var /lib/jenkins, JENKINS_SERVER_COOKIE:c131d783e95d951e, JENKINS_URL:http://vm-kwirth-swarm182-xenial:8080/, JOB_BASE_NAME:JENKINS-64707_ws_root, JOB_DISPLAY_URL:http://vm-kwirth-swarm182-xenial:8080/job/JENKINS-64707_ws_root/display/redirect, JOB_NAME:JENKINS-64707_ws_root, JOB_URL:http://vm-kwirth-swarm182-xenial:8080/job/JENKINS-64707_ws_root/, P4_CHANGELIST:215349, P4_CLIENT:JENKINS-64707-ws, P4_PORT:vm-kwirth-swarm182-xenial:1666, P4_ROOT:E:\filestore\Jenkins\outside_Windows10Slave22, P4_TICKET:9058B5D7DC53D417EDFEC9E63AB39BBB, P4_USER: super , P4CLIENT:JENKINS-64707-ws, RUN_CHANGES_DISPLAY_URL:http://vm-kwirth-swarm182-xenial:8080/job/JENKINS-64707_ws_root/16/display/redirect?page=changes, RUN_DISPLAY_URL:http://vm-kwirth-swarm182-xenial:8080/job/JENKINS-64707_ws_root/16/display/redirect] Also note that originally my workspace root was 'E:\filestore\test' but it has now been changed by P4Jenkins to match the value of 'customWorkspace': 12:17:42 E:\filestore\Jenkins\outside_Windows10Slave22>p4 -p vm-kwirth-swarm182-xenial:1666 -u super -P 9058B5D7DC53D417EDFEC9E63AB39BBB -c jenkins-Windows10-JENKINS-64707_ws_root-0-thescript client -o 12:17:43 # A Perforce Client Specification. 12:17:43 # ...cut... 12:17:43 12:17:43 Client: jenkins-Windows10-JENKINS-64707_ws_root-0-thescript 12:17:43 12:17:43 Update: 2021/01/27 11:59:06 12:17:43 12:17:43 Access: 2021/01/27 12:17:30 12:17:43 12:17:43 Owner: super 12:17:43 12:17:43 Root: E:\filestore\Jenkins\outside_Windows10Slave22 *** THIS HAS BEEN OVERWRITTEN *** 12:17:43 12:17:43 Options: noallwrite clobber nocompress unlocked nomodtime normdir 12:17:43 12:17:43 SubmitOptions: submitunchanged 12:17:43 12:17:43 LineEnd: local 12:17:43 12:17:43 View: 12:17:43 //depot/JENKINS-64707_ws_root/... //jenkins-Windows10-JENKINS-64707_ws_root-0-thescript/...        

          Karl Wirth added a comment -

          Closing for now. If this is still a problem please send an email to support@perforce.com and we can dig deeper into why 'ws/customWorkspace' does not work for you.

          Karl Wirth added a comment - Closing for now. If this is still a problem please send an email to support@perforce.com and we can dig deeper into why 'ws/customWorkspace' does not work for you.

          Andrew Barber added a comment -

          Agree with poster.  The name P4_ROOT is misleading as it is not what perforce returns as the client root.  It is in fact the same as WORKSPACE, so why create another variable for it? 

          Andrew Barber added a comment - Agree with poster.  The name P4_ROOT is misleading as it is not what perforce returns as the client root.  It is in fact the same as WORKSPACE, so why create another variable for it? 

          Karl Wirth added a comment -

          Hi ajbarber - They should be the same in most cases but suddenly I have changed my mind on this job. The documentation states:

           

          P4_ROOT - The Perforce client workspace root path. 

           

          In my testing, the one case that P4_ROOT does not equal WORKSPACE (and the  client 'Root:'  is not overwritten by p4-plugin) is when it's a 'Static' workspace. In that case the workspace can only run on the Jenkins master but in my testing does save outside the workspace root.

          Reopening this bug.

           

          Reproduction Steps:

          (1) Create a Perforce workspace called 'static' with no 'Host:'  field and 'Root:' set to '/var/lib/jenkins/workspace/STATIC'.

          (2) Create a pipeline job called 'Fixed_workspace' with following code:

           

          node {
              stage('Preparation') { // for display purposes
                 checkout perforce(credential: 'JenkinsMaster', populate: autoClean(delete: true, modtime: false, parallel: [enable: false, minbytes: '1024', minfiles: '1', threads: '4'], pin: '', quiet: false, replace: true, tidy: false), workspace: staticSpec(charset: 'none', name: 'static', pinHost: false))
                 sh('env')
                 sh('pwd')
              }
          } 

          (3) Run job and look at console logs:

           

          It syncs to the original 'Root:' ('/var/lib/jenkins/workspace/STATIC'):

           

          P4 Task: cleaning workspace to match have list.
          ... p4 reconcile -f -w /var/lib/jenkins/workspace/STATIC/... +
          //depot/BuildBot/f2435#1 - /var/lib/jenkins/workspace/STATIC/project1/main/BuildBot/f2435 refreshed
          //depot/BuildBot/f2436#1 - /var/lib/jenkins/workspace/STATIC/project1/main/BuildBot/f2436 refreshed 

          but when you check the variables in the same job  P4_ROOT is set to the WORKSPACE:

           

           

          [Fixed_workspace] $ /bin/sh -xe /tmp/jenkins4743276461981135487.sh
          ...cut...
          WORKSPACE=/var/lib/jenkins/workspace/Fixed_workspace
          ...cut...
          P4_ROOT=/var/lib/jenkins/workspace/Fixed_workspace 
          ...cut...

          Expected result:

          P4_ROOT = /var/lib/jenkins/workspace/STATIC

           

           

          Karl Wirth added a comment - Hi ajbarber - They should be the same in most cases but suddenly I have changed my mind on this job. The documentation states:   P4_ROOT - The Perforce client workspace root path.   In my testing, the one case that P4_ROOT does not equal WORKSPACE (and the  client 'Root:'  is not overwritten by p4-plugin) is when it's a 'Static' workspace. In that case the workspace can only run on the Jenkins master but in my testing does save outside the workspace root. Reopening this bug.   Reproduction Steps: (1) Create a Perforce workspace called 'static' with no 'Host:'  field and 'Root:' set to '/var/lib/jenkins/workspace/STATIC'. (2) Create a pipeline job called 'Fixed_workspace' with following code:   node {     stage( 'Preparation' ) { // for display purposes        checkout perforce(credential: 'JenkinsMaster' , populate: autoClean(delete: true , modtime: false , parallel: [enable: false , minbytes: '1024' , minfiles: '1' , threads: '4' ], pin: '', quiet: false , replace: true , tidy: false ), workspace: staticSpec(charset: ' none ', name: ' static ', pinHost: false ))        sh( 'env' )        sh( 'pwd' )     } } (3) Run job and look at console logs:   It syncs to the original 'Root:' ('/var/lib/jenkins/workspace/STATIC'):   P4 Task: cleaning workspace to match have list. ... p4 reconcile -f -w / var /lib/jenkins/workspace/STATIC/... + //depot/BuildBot/f2435#1 - / var /lib/jenkins/workspace/STATIC/project1/main/BuildBot/f2435 refreshed //depot/BuildBot/f2436#1 - / var /lib/jenkins/workspace/STATIC/project1/main/BuildBot/f2436 refreshed but when you check the variables in the same job  P4_ROOT is set to the WORKSPACE:     [Fixed_workspace] $ /bin/sh -xe /tmp/jenkins4743276461981135487.sh ...cut... WORKSPACE=/ var /lib/jenkins/workspace/Fixed_workspace ...cut... P4_ROOT=/ var /lib/jenkins/workspace/Fixed_workspace ...cut... Expected result: P4_ROOT = /var/lib/jenkins/workspace/STATIC    

          Karl Wirth added a comment -

          Tested in most recent version and for a static workspace the 'Root:' is not ovewrwritten so P4_ROOT should match 'Root:'.

          Karl Wirth added a comment - Tested in most recent version and for a static workspace the 'Root:' is not ovewrwritten so P4_ROOT should match 'Root:'.

            Unassigned Unassigned
            plawniczuk Piotr Ławniczuk
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: