Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-60358

Support Root=null value in Template workspace - Currently it is overridden, failing sync

    • Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Major Major
    • p4-plugin
    • Windows

      P4 Windows use Root:null to support mapping sync to different drives. 
      this value is always overwritten when using p4 plugin, thus failing the sync. 
      p4 sync should use .../@ instead, and the workspace Root should not change. 

          [JENKINS-60358] Support Root=null value in Template workspace - Currently it is overridden, failing sync

          Karl Wirth added a comment -

          Hi shai_shap - Thanks for the feedback.

          Can you give me an example of why your build systems are using different drives for the Perforce data so that I can pass that to the developers. Please also give me screen shots of your job definition and/or Jenkinsfile.

          Jenkins usually controls where we should sync the files which is why p4-plugin is designed to overwrite the root with the location provided by Jenkins. However this does not work well with the model of different files on different Windows drive or fixed locations on the disk.

          To work around the problem you could use multiple 'p4sync' or 'checkout' steps to each drive in a Jenkinsfile but would need to change 'ws' path before each sync.  Another way is to use P4Groovy or a batch file and 'p4.exe' but you would lose the power of polling and change reporting.

           

          Reproduction Steps:

          (1) Setup a build slave on Windows.

          (2) Create a workspace with a null root. For example:

          Client:	ws_root_set
          Root:	null
          View:
          	//depot/Project1/... //ws_root_set/C:/filestore/ws/JENKINS/...

          This would sync the files from '//depot/project1/...' to 'C:\filestore\ws\JENKINS'.

          (3) If you use the workspace behavior 'Static' we will try to sync the path 'null/...'. If you use Template the '\C:\' would be inserted into the path. Each way will cause a bad build.

          See related job JENKINS-47005.

           

          Karl Wirth added a comment - Hi shai_shap - Thanks for the feedback. Can you give me an example of why your build systems are using different drives for the Perforce data so that I can pass that to the developers. Please also give me screen shots of your job definition and/or Jenkinsfile. Jenkins usually controls where we should sync the files which is why p4-plugin is designed to overwrite the root with the location provided by Jenkins. However this does not work well with the model of different files on different Windows drive or fixed locations on the disk. To work around the problem you could use multiple 'p4sync' or 'checkout' steps to each drive in a Jenkinsfile but would need to change 'ws' path before each sync.  Another way is to use P4Groovy or a batch file and 'p4.exe' but you would lose the power of polling and change reporting.   Reproduction Steps: (1) Setup a build slave on Windows. (2) Create a workspace with a null root. For example: Client: ws_root_set Root: null View: //depot/Project1/... //ws_root_set/C:/filestore/ws/JENKINS/... This would sync the files from '//depot/project1/...' to 'C:\filestore\ws\JENKINS'. (3) If you use the workspace behavior 'Static' we will try to sync the path 'null/...'. If you use Template the '\C:\' would be inserted into the path. Each way will cause a bad build. See related job JENKINS-47005 .  

          Guy Catz added a comment -

          Hi Karl,

          I'm Guy, working Shai Shapira on that.

          To answer your question "....why your build systems are using different drives for the Perforce data".

          Our Windows build is comprised of a large amount of very small files (10^5). To make the cleanup faster after a build, we simply format that disk.

          Moreover, a big chunk of the sources (~ 30GB) are SDKs that are rarely being changed. Instead of forced-syncing it on every build, we keep it in a separated drive, (non-forced) sync it and logically combined both drive into a one partition.

          For that, we keep the root null.

          The job below is a test for that matter.

          Could u explain more about the workaround? How do we achieve that while still using the p4 plugin?

          As a solution, can you introduce a checkbox "override root path" (default checked)?

          Thanks

          Guy Catz added a comment - Hi Karl, I'm Guy, working Shai Shapira on that. To answer your question "....why your build systems are using different drives for the Perforce data". Our Windows build is comprised of a large amount of very small files (10^5). To make the cleanup faster after a build, we simply format that disk. Moreover, a big chunk of the sources (~ 30GB) are SDKs that are rarely being changed. Instead of forced-syncing it on every build, we keep it in a separated drive, (non-forced) sync it and logically combined both drive into a one partition. For that, we keep the root null. The job below is a test for that matter. Could u explain more about the workaround? How do we achieve that while still using the p4 plugin? As a solution, can you introduce a checkbox "override root path" (default checked)? Thanks

          Karl Wirth added a comment -

          Hi guy_catz, Yes I'll put the checkbox for null root in as an enhancement request.

          The workaround is to use a Pipeline job and do separate sync's for each drive.

          Below is an example of a piece of code that runs on a slave called 'Windows10' and syns code from '//depot/folder1/...' to C:\Jenkins1 and '//depot/folder2/...' to 'E:\Jenkins2':

           

          pipeline {
            
            agent { label "Windows10" }
            
            stages {
              stage("Sync to C:\\") {
                steps {
                  ws("C:\\Jenkins1") {
                     p4sync charset: 'none', credential: 'JenkinsMaster', populate: autoClean(delete: true, modtime: false, parallel: [enable: false, minbytes: '1024', minfiles: '1', threads: '4'], pin: '', quiet: false, replace: true, tidy: false), source: depotSource('//depot/folder1/...')
                  }
                }
              }
              stage("Sync to D:\\") {
                steps {
                  ws("E:\\Jenkins2") {
                     p4sync charset: 'none', credential: 'JenkinsMaster', populate: autoClean(delete: true, modtime: false, parallel: [enable: false, minbytes: '1024', minfiles: '1', threads: '4'], pin: '', quiet: false, replace: true, tidy: false), source: depotSource('//depot/folder2/...')
                  }
                }
              }
            }
          }
          

           

           

           

          Karl Wirth added a comment - Hi guy_catz , Yes I'll put the checkbox for null root in as an enhancement request. The workaround is to use a Pipeline job and do separate sync's for each drive. Below is an example of a piece of code that runs on a slave called 'Windows10' and syns code from '//depot/folder1/...' to C:\Jenkins1 and '//depot/folder2/...' to 'E:\Jenkins2':   pipeline { agent { label "Windows10" } stages { stage( "Sync to C:\\" ) { steps { ws( "C:\\Jenkins1" ) { p4sync charset: 'none' , credential: 'JenkinsMaster' , populate: autoClean(delete: true , modtime: false , parallel: [enable: false , minbytes: '1024' , minfiles: '1' , threads: '4' ], pin: '', quiet: false , replace: true , tidy: false ), source: depotSource(' //depot/folder1/...') } } } stage( "Sync to D:\\" ) { steps { ws( "E:\\Jenkins2" ) { p4sync charset: 'none' , credential: 'JenkinsMaster' , populate: autoClean(delete: true , modtime: false , parallel: [enable: false , minbytes: '1024' , minfiles: '1' , threads: '4' ], pin: '', quiet: false , replace: true , tidy: false ), source: depotSource(' //depot/folder2/...') } } } } }      

          Shai Shapira added a comment -

          Thanks, but this breaks current simplicity for allowing multi-config management as well (using Jenkins multi config add-on) - Use of pipeline job affects UI simplification for users, which we are currently looking for.

          Shai Shapira added a comment - Thanks, but this breaks current simplicity for allowing multi-config management as well (using Jenkins multi config add-on) - Use of pipeline job affects UI simplification for users, which we are currently looking for.

          Shai Shapira added a comment -

          Any Update?

          Shai Shapira added a comment - Any Update?

          Karl Wirth added a comment -

          Hi shai_shap,

          This is still in the backlog but you commenting on it will mark it as an active job for the developers the next time they review the backlog.

          Your previous comments suggests that the reason you are not using the workaround is it is more confusing for users? Is that correct or did I misread it?

          Regards,

          Karl

          Karl Wirth added a comment - Hi shai_shap , This is still in the backlog but you commenting on it will mark it as an active job for the developers the next time they review the backlog. Your previous comments suggests that the reason you are not using the workaround is it is more confusing for users? Is that correct or did I misread it? Regards, Karl

            Unassigned Unassigned
            shai_shap Shai Shapira
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: