Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-57314

P4 Sync to label after sync to head without deleting non labeled files

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • p4-plugin
    • linux

      I've created a perforce workspace and I've sync it to the head version.

      However I would like to sync the same workspace to a label but without deleting files that are not in the label. Like running the command p4 sync <workspace>/... @=label . 

      I've tried many combinations of the populate command but all of them have run something like  p4 sync <arguments><workspace>/... @label 

      Is there a way to run the p4sync command on jenkins so that it run a p4 sync ** @= command?*

       

      Thanks!

          [JENKINS-57314] P4 Sync to label after sync to head without deleting non labeled files

          Karl Wirth added a comment -

          Hi citylife4 - Not sure so I will try it out here and get back to you with my findings.

          Karl Wirth added a comment - Hi citylife4 - Not sure so I will try it out here and get back to you with my findings.

          Karl Wirth added a comment -

          Hi citylife4. I have not been able to find a way to do this so will mark this as an enhancement request for the developers. The workaround would be to do the whole sync using P4Groovy:

             https://www.perforce.com/manuals/jenkins/Content/P4Jenkins/p4groovy.html?Highlight=P4Groovy

          You could then specify the command line you need for the sync.

          Karl Wirth added a comment - Hi citylife4 . I have not been able to find a way to do this so will mark this as an enhancement request for the developers. The workaround would be to do the whole sync using P4Groovy:    https://www.perforce.com/manuals/jenkins/Content/P4Jenkins/p4groovy.html?Highlight=P4Groovy You could then specify the command line you need for the sync.

          Jose Valverde added a comment -

          Hi p4karl .

          Thanks for you feedback,

          In the meanwhile, I'll try to sync my workspace as you've instructed!

          Jose Valverde added a comment - Hi p4karl . Thanks for you feedback, In the meanwhile, I'll try to sync my workspace as you've instructed!

          Jose Valverde added a comment -

          Hi p4karl,

          I'm trying to change my job to run a P4Groovy, and I've created an example as in the link you've shared, and i'm getting the following error:

           

          [Pipeline] End of Pipeline
          ERROR: P4: Task Exception: Invalid credentials
          

           

          This is my test code:

           

          node("my_node") {
              // Define workspace
              def ws = [$class: 'StreamWorkspaceImpl', 
              charset: 'none', format: 'jenkins-${JOB_NAME}', 
              pinHost: false, streamName: '//my_workspace']// Create object
              def p4 = p4 (credential: 'my_client', workspace: manualSpec(charset: 'none', cleanup: false, name: 'jenkins-${NODE_NAME}-${JOB_NAME}-${EXECUTOR_NUMBER}', pinHost: false, spec: clientSpec(allwrite: false, backup: true, changeView: '', clobber: true, compress: false, line: 'LOCAL', locked: false, modtime: false, rmdir: false, serverID: '', streamName: '', type: 'WRITABLE', view: 'my_workspace')))
             p4.run('sync','...')
          }
          

           

          Please note that "my_client" is the same using in other working jobs.

          Is there any additional configuration needed on my job, rather then what is on the shared link (de-select groovy sandbox) ?

           

          Thanks for the help!

           

          Jose Valverde added a comment - Hi p4karl , I'm trying to change my job to run a P4Groovy, and I've created an example as in the link you've shared, and i'm getting the following error:   [Pipeline] End of Pipeline ERROR: P4: Task Exception: Invalid credentials   This is my test code:   node( "my_node" ) { // Define workspace def ws = [$class: 'StreamWorkspaceImpl' , charset: 'none' , format: 'jenkins-${JOB_NAME}' , pinHost: false , streamName: ' //my_workspace' ]// Create object def p4 = p4 (credential: 'my_client' , workspace: manualSpec(charset: 'none' , cleanup: false , name: 'jenkins-${NODE_NAME}-${JOB_NAME}-${EXECUTOR_NUMBER}' , pinHost: false , spec: clientSpec(allwrite: false , backup: true , changeView: '', clobber: true , compress: false , line: ' LOCAL ', locked: false , modtime: false , rmdir: false , serverID: ' ', streamName: ' ', type: ' WRITABLE ', view: ' my_workspace'))) p4.run( 'sync' , '...' ) }   Please note that "my_client" is the same using in other working jobs. Is there any additional configuration needed on my job, rather then what is on the shared link (de-select groovy sandbox) ?   Thanks for the help!  

          Karl Wirth added a comment -

          Hi citylife4,

          Sorry 'credential' is a typo. That should be your Jenkins credential not the workspace name. For example:

           def p4 = p4 (credential: 'myJenkinsCrentialName',workspace: ...
          

          Karl Wirth added a comment - Hi citylife4 , Sorry 'credential' is a typo. That should be your Jenkins credential not the workspace name. For example: def p4 = p4 (credential: 'myJenkinsCrentialName' ,workspace: ...

          Jose Valverde added a comment -

          Hi p4karl,

          My mistake, indeed I have my jenkins credentials on the fcredentials options not my workspace name.

          Jose Valverde added a comment - Hi p4karl , My mistake, indeed I have my jenkins credentials on the fcredentials options not my workspace name.

          Hi p4karl,

          I've also tried this kind of construction.

          In my case I have created a shared library (my_groovy_lib). Inside the vars/ folder I've placed a call p4SyncLabel.groovy (see in attachmentp4SyncLabel.groovy

          Then inside the main pipe I call:

          #!/usr/bin/env groovy  
          @Library('my_groovy_lib') _
          // Pipeline steering
          pipeline {
           agent {
              label 'my_slave_0'
           }
           options {
              timestamps()
           }
           stages {
              stage('Test') {
                 steps{
                    echo 'Test'
                    script{
                       p4SyncLabel('my_p4_ticket_id', "jenkins_${NODE_NAME}_${JOB_BASE_NAME}_${EXECUTOR_NUMBER}",
                                    "//depot/... //jenkins_${NODE_NAME}_${JOB_BASE_NAME}_${EXECUTOR_NUMBER}/...",
                                   'head', true, false )
                    }
                  }
                }
             }
          }
          

           

           

          I'm getting the following output:

           

          Started by user Frederico Pratas
          Running in Durability level: MAX_SURVIVABILITY
          Loading library my_groovy_lib@master
          ... p4 client -o jenkins_shared_lib_master_example +
          ... p4 info +
          ... p4 info +
          ... p4 client -o jenkins_shared_lib_master_example +
          ... No change in client detected.
          ... p4 counter change +
          ... p4 changes -m1 -ssubmitted //jenkins_shared_lib_master___ +
          Building on Node: master
          ... p4 client -o jenkins_shared_lib_master_example +
          ... p4 info +
          ... p4 info +
          ... p4 client -o jenkins_shared_lib_master_example +
          ... No change in client detected.
          P4 Task: establishing connection.
          ... server: XXXXX:1666
          ... node: XXXXX
          P4 Task: reverting all pending and shelved revisions.
          ... p4 revert /var/lib/jenkins/workspace/example%40libs/sn___ +
          ... rm [abandoned files]
          duration: (845ms)
          P4 Task: cleaning workspace to match have list.
          ... p4 reconcile -w -f /var/lib/jenkins/workspace/example%___ +
          duration: 0m 1s
          P4 Task: syncing files at change: XXXXX
          ... p4 sync -q /var/lib/jenkins/workspace/example%40libs/s___ +
          duration: 0m 4s
          P4 Task: saving built changes.
          Found last change XXXXX on syncID jenkins_shared_lib_NODE_NAME_example
          ... p4 client -o jenkins_shared_lib_master_example +
          ... p4 info +
          ... p4 info +
          ... p4 client -o jenkins_shared_lib_master_example +
          ... No change in client detected.
          ... p4 changes -m20 //jenkins_shared_lib_master_example/..___ +
          ... p4 changes -l -m1 @XXXXX +
          ... p4 user -o pratas +
          ... p4 describe -s -m51 XXXXX +
          ... p4 fixes -cXXXXX +
          ... done
          [Pipeline] Start of Pipeline
          [Pipeline] node
          Running on my_slave_0 in /users/jenkins/workspace/example
          [Pipeline] {
          [Pipeline] timestamps
          [Pipeline] {
          [Pipeline] stage
          [Pipeline] { (Test)
          [Pipeline] echo
          09:45:20 Test
          [Pipeline] script
          [Pipeline] {
          [Pipeline] p4
          [Pipeline] }
          [Pipeline] // script
          [Pipeline] }
          [Pipeline] // stage
          [Pipeline] }
          [Pipeline] // timestamps
          [Pipeline] }
          [Pipeline] // node
          [Pipeline] End of Pipeline
          ERROR: P4: Task Exception: Invalid credentials
          Finished: FAILURE
          

           

           

          I'm using the same credential that is used to load the library from p4, which as you can see works fine.

          Any ideas what is wrong in the flow?

           

          Thanks,

          Frederico

           

           

          Frederico Pratas added a comment - Hi p4karl , I've also tried this kind of construction. In my case I have created a shared library ( my_groovy_lib ). Inside the vars/ folder I've placed a call p4SyncLabel.groovy (see in attachment p4SyncLabel.groovy Then inside the main pipe I call: #!/usr/bin/env groovy   @Library( 'my_groovy_lib' ) _ // Pipeline steering pipeline { agent { label 'my_slave_0' } options { timestamps() } stages { stage( 'Test' ) { steps{ echo 'Test' script{ p4SyncLabel( 'my_p4_ticket_id' , "jenkins_${NODE_NAME}_${JOB_BASE_NAME}_${EXECUTOR_NUMBER}" , " //depot/... //jenkins_${NODE_NAME}_${JOB_BASE_NAME}_${EXECUTOR_NUMBER}/..." , 'head' , true , false ) } } } } }     I'm getting the following output:   Started by user Frederico Pratas Running in Durability level: MAX_SURVIVABILITY Loading library my_groovy_lib@master ... p4 client -o jenkins_shared_lib_master_example + ... p4 info + ... p4 info + ... p4 client -o jenkins_shared_lib_master_example + ... No change in client detected. ... p4 counter change + ... p4 changes -m1 -ssubmitted //jenkins_shared_lib_master___ + Building on Node: master ... p4 client -o jenkins_shared_lib_master_example + ... p4 info + ... p4 info + ... p4 client -o jenkins_shared_lib_master_example + ... No change in client detected. P4 Task: establishing connection. ... server: XXXXX:1666 ... node: XXXXX P4 Task: reverting all pending and shelved revisions. ... p4 revert / var /lib/jenkins/workspace/example%40libs/sn___ + ... rm [abandoned files] duration: (845ms) P4 Task: cleaning workspace to match have list. ... p4 reconcile -w -f / var /lib/jenkins/workspace/example%___ + duration: 0m 1s P4 Task: syncing files at change: XXXXX ... p4 sync -q / var /lib/jenkins/workspace/example%40libs/s___ + duration: 0m 4s P4 Task: saving built changes. Found last change XXXXX on syncID jenkins_shared_lib_NODE_NAME_example ... p4 client -o jenkins_shared_lib_master_example + ... p4 info + ... p4 info + ... p4 client -o jenkins_shared_lib_master_example + ... No change in client detected. ... p4 changes -m20 //jenkins_shared_lib_master_example/..___ + ... p4 changes -l -m1 @XXXXX + ... p4 user -o pratas + ... p4 describe -s -m51 XXXXX + ... p4 fixes -cXXXXX + ... done [Pipeline] Start of Pipeline [Pipeline] node Running on my_slave_0 in /users/jenkins/workspace/example [Pipeline] { [Pipeline] timestamps [Pipeline] { [Pipeline] stage [Pipeline] { (Test) [Pipeline] echo 09:45:20 Test [Pipeline] script [Pipeline] { [Pipeline] p4 [Pipeline] } [Pipeline] // script [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // timestamps [Pipeline] } [Pipeline] // node [Pipeline] End of Pipeline ERROR: P4: Task Exception: Invalid credentials Finished: FAILURE     I'm using the same credential that is used to load the library from p4, which as you can see works fine. Any ideas what is wrong in the flow?   Thanks, Frederico    

          Jose Valverde added a comment - - edited

          Hi p4karl,

          Do you have any idea on why we are having this problem?

          Thanks for the help,

          Jose

          Jose Valverde added a comment - - edited Hi  p4karl , Do you have any idea on why we are having this problem? Thanks for the help, Jose

          Karl Wirth added a comment -

          Hi citylife4 - Sorry I've been on vacation. I'll take a look and get back to you.

          Karl Wirth added a comment - Hi citylife4 - Sorry I've been on vacation. I'll take a look and get back to you.

          Chris Carr added a comment - - edited

          I'm hitting the same problem - I've been experimenting with using the P4 object to sync to a small subset of the stream we are building on, on the master. Using p4sync with the same credentials works correctly, but creating a p4 object and then syncing fails with "ERROR: P4: Task Exception: Invalid credentials". Unfortunately it's not practical to fully sync to our streams on the master, as they are quite large.

          Chris Carr added a comment - - edited I'm hitting the same problem - I've been experimenting with using the P4 object to sync to a small subset of the stream we are building on, on the master. Using p4sync with the same credentials works correctly, but creating a p4 object and then syncing fails with "ERROR: P4: Task Exception: Invalid credentials". Unfortunately it's not practical to fully sync to our streams on the master, as they are quite large.

          Karl Wirth added a comment -

          Hi both. I hit this invalid credentials when there was a typo in my credential ID. Can you please try the following simple code test to see if it works for you. Please replace 'JenkinsMaster' with the name of your Perforce credential ID and '//depot/Jam/...' with a small path in your depot:

          def p4SyncLabel(String credential, String ws_name, String ws_view, String label, boolean append, boolean force){
                  echo "In function ..."
                  def ws = [$class: 'ManualWorkspaceImpl', name: ws_name, spec: [view: ws_view]]
                  def p4 = p4(credential: credential, workspace: ws)
                  p4.run('sync')
                  echo "... function end"
          }
          
          pipeline {
           agent {
              label 'master'
           }
           options {
              timestamps()
           }
           stages {
              stage('Test') {
                 steps{
                    echo 'Test'
                    script{
                       p4SyncLabel('JenkinsMaster', "jenkins_${NODE_NAME}_${JOB_BASE_NAME}_${EXECUTOR_NUMBER}",
                                    "//depot/Jam/... //jenkins_${NODE_NAME}_${JOB_BASE_NAME}_${EXECUTOR_NUMBER}/...",
                                   'head', true, false )
                    }
                  }
                }
             }
          }
          

           

          Karl Wirth added a comment - Hi both. I hit this invalid credentials when there was a typo in my credential ID. Can you please try the following simple code test to see if it works for you. Please replace 'JenkinsMaster' with the name of your Perforce credential ID and '//depot/Jam/...' with a small path in your depot: def p4SyncLabel( String credential, String ws_name, String ws_view, String label, boolean append, boolean force){ echo "In function ..." def ws = [$class: 'ManualWorkspaceImpl' , name: ws_name, spec: [view: ws_view]] def p4 = p4(credential: credential, workspace: ws) p4.run( 'sync' ) echo "... function end" } pipeline { agent { label 'master' } options { timestamps() } stages { stage( 'Test' ) { steps{ echo 'Test' script{ p4SyncLabel( 'JenkinsMaster' , "jenkins_${NODE_NAME}_${JOB_BASE_NAME}_${EXECUTOR_NUMBER}" , " //depot/Jam/... //jenkins_${NODE_NAME}_${JOB_BASE_NAME}_${EXECUTOR_NUMBER}/..." , 'head' , true , false ) } } } } }  

          Chris Carr added a comment - - edited

          Hi . I ran the job, and it failed with the output below.

          I double checked the credentials name, and it was correct (and it's using the same credentials with a P4Sync call that has no problems). We're on version 1.9.6 of the Jenkins plugin, but plan on updating to 1.10.0 very shortly.

           

          -------

          Started by user ccarr
          Running in Durability level: MAX_SURVIVABILITY
          [Pipeline] Start of Pipeline
          [Pipeline] node
          Running on Jenkins in /var/lib/jenkins/workspace/Project/Test_P4Jenkins
          [Pipeline] {
          [Pipeline] timestamps
          [Pipeline] {
          [Pipeline] stage
          [Pipeline] { (Test)
          [Pipeline] echo
          18:37:36 Test
          [Pipeline] script
          [Pipeline]

          { [Pipeline] echo 18:37:36 In function ... [Pipeline] p4 [Pipeline] }

          [Pipeline] // script
          [Pipeline] }
          [Pipeline] // stage
          [Pipeline] }
          [Pipeline] // timestamps
          [Pipeline] }
          [Pipeline] // node
          [Pipeline] End of Pipeline
          ERROR: P4: Task Exception: Invalid credentials
          Finished: FAILURE

          Chris Carr added a comment - - edited Hi . I ran the job, and it failed with the output below. I double checked the credentials name, and it was correct (and it's using the same credentials with a P4Sync call that has no problems). We're on version 1.9.6 of the Jenkins plugin, but plan on updating to 1.10.0 very shortly.   ------- Started by user ccarr Running in Durability level: MAX_SURVIVABILITY [Pipeline] Start of Pipeline [Pipeline] node Running on Jenkins in /var/lib/jenkins/workspace/Project/Test_P4Jenkins [Pipeline] { [Pipeline] timestamps [Pipeline] { [Pipeline] stage [Pipeline] { (Test) [Pipeline] echo 18:37:36 Test [Pipeline] script [Pipeline] { [Pipeline] echo 18:37:36 In function ... [Pipeline] p4 [Pipeline] } [Pipeline] // script [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // timestamps [Pipeline] } [Pipeline] // node [Pipeline] End of Pipeline ERROR: P4: Task Exception: Invalid credentials Finished: FAILURE

          Jose Valverde added a comment -

          Hi Karl!

           

          I'm having the same issue as mentioned above.

          And we are also using version 1.9.6!

           
           
          Running in Durability level: MAX_SURVIVABILITY[Pipeline] Start of Pipeline[Pipeline] node....[Pipeline] {[Pipeline] timestamps[Pipeline] {[Pipeline] stage[Pipeline] { (Test)[Pipeline] echo*09:20:51* Test[Pipeline] script[Pipeline]

          {[Pipeline] echo*09:20:51* In function ...[Pipeline] p4[Pipeline] }

          [Pipeline] // script[Pipeline] }[Pipeline] // stage[Pipeline] }[Pipeline] // timestamps[Pipeline] }[Pipeline] // node[Pipeline] End of PipelineERROR: P4: Task Exception: Invalid credentials
          Finished: FAILURE

          Jose Valverde added a comment - Hi Karl!   I'm having the same issue as mentioned above. And we are also using version 1.9.6!     Running in Durability level: MAX_SURVIVABILITY [Pipeline] Start of Pipeline [Pipeline] node.... [Pipeline] { [Pipeline] timestamps [Pipeline] { [Pipeline] stage [Pipeline] { (Test) [Pipeline] echo*09:20:51* Test [Pipeline] script [Pipeline] {[Pipeline] echo*09:20:51* In function ...[Pipeline] p4[Pipeline] } [Pipeline] // script [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // timestamps [Pipeline] } [Pipeline] // node [Pipeline] End of PipelineERROR: P4: Task Exception: Invalid credentials Finished: FAILURE

          Chris Carr added a comment -

          We've updated the p4 plugin to 1.10.1, and the issue still persists. I also tried adding a p4sync() call, using the same credentials string to do a preview sync in the same job. It worked fine, so the credentials are definitely okay.

          Chris Carr added a comment - We've updated the p4 plugin to 1.10.1, and the issue still persists. I also tried adding a p4sync() call, using the same credentials string to do a preview sync in the same job. It worked fine, so the credentials are definitely okay.

          Karl Wirth added a comment -

          Hi chriscarr, citylife4

          I'd like to setup a webex with one of you to investigate further. Can one of you comment in this case then send an email to support@perforce.com so I can arrange that.

           

          Note: 1.0.2 just shipped today but as far as I know it has no credential related fixes.

           

          Thanks in advance.

           

          Karl

          Karl Wirth added a comment - Hi chriscarr , citylife4 I'd like to setup a webex with one of you to investigate further. Can one of you comment in this case then send an email to support@perforce.com so I can arrange that.   Note: 1.0.2 just shipped today but as far as I know it has no credential related fixes.   Thanks in advance.   Karl

          Karl Wirth added a comment -

          Hi chriscarr, citylife4,

          I just wanted to check in again to see if either of you are available for a webex during UK working hours?

          Karl Wirth added a comment - Hi chriscarr , citylife4 , I just wanted to check in again to see if either of you are available for a webex during UK working hours?

          Jose Valverde added a comment -

          Hi p4karl,

           

          Are you available to schedule a webex for next Monday (July 22) at 4PM UK time?

          I'm going to send the email to support@perforce.com now! 

           

          Thanks!

          Jose Valverde added a comment - Hi p4karl ,   Are you available to schedule a webex for next Monday (July 22) at 4PM UK time? I'm going to send the email to support@perforce.com  now!    Thanks!

          Karl Wirth added a comment -

          Hi citylife4 - Perfect. Thanks.

          Karl Wirth added a comment - Hi citylife4 - Perfect. Thanks.

          Frederico Pratas added a comment - - edited

          Hi,

          After some digging with joel_brown and p4karl we (citylife4 and I) managed to isolate and reproduce the credential issue.

          Another bug has been created to track this JENKINS-58745

          chriscarr you may want to follow up in case the issue matches your setup as well.

          Thanks,

          Frederico

          Frederico Pratas added a comment - - edited Hi, After some digging with  joel_brown  and p4karl  we ( citylife4 and I) managed to isolate and reproduce the credential issue. Another bug has been created to track this  JENKINS-58745 chriscarr you may want to follow up in case the issue matches your setup as well. Thanks, Frederico

            p4karl Karl Wirth
            citylife4 Jose Valverde
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: