• Icon: New Feature New Feature
    • Resolution: Unresolved
    • Icon: Major Major
    • p4-plugin

      One of Perforce customers strongly requests Buildfarm server support, specially for the following workflow;

      1. Create new Perforce workspaces
      2. Sync files to the Perforce workspaces
      3. Build
      4. Delete the Perforce workspaces

          [JENKINS-52088] Support for Perforce Buildfarm server

          Karl Wirth added a comment -

          Note: The above scenario is the only workflow that could work with build farm replicas. Anything that requires 'p4 revert' or  'p4 reconcile' to clean up the workspace or tries to publish a build/results to the Perforce server would fail.

           

          Therefore only the 'Populate options' called 'Sync only' could be supported without checking 'Revert any open or unshelved files'.

           

          Karl Wirth added a comment - Note : The above scenario is the only workflow that could work with build farm replicas. Anything that requires 'p4 revert' or  'p4 reconcile' to clean up the workspace or tries to publish a build/results to the Perforce server would fail.   Therefore only the 'Populate options' called 'Sync only' could be supported without checking 'Revert any open or unshelved files'.  

          Kevin Park added a comment -

          I am sorry that I have missed mentioning buildfarm replica support in Multibranch Pipeline project.

           

          With the following multibranch pipeline project configuration, and the following syntax in Jenkinsfile;

          [configuration]

           
          [Jenkinsfile]

          populate: syncOnly(

          force: true,

          have: false,

          modtime: false,

          parallel: [enable: false, minbytes: '1024', minfiles: '1', threads: '4'], pin: '', quiet: false, revert: false),

           

          'p4 revert' is still running. And, 'p4 client -d' is not running, which is running when connected to an edge server.
          P4 Task: remove client: jenkins-master-edge_multi-kpark_bui-kpark_bui-0
          ... p4 client -d jenkins-master-edge_multi-kpark_bui-kpark_bui-0 -
           

          Also there is "ERROR: Could not do lightweight checkout, falling back to heavyweight" error as well.

           

          Full Jenkins log is as below. 
          Console Output
          Started by user jenkins
          ... p4 dirs //test/jenkins/* +
          ... p4 changes -m1 //test/jenkins/DSP/... +
          ERROR: Could not do lightweight checkout, falling back to heavyweight
          com.perforce.p4java.exception.RequestException: //jenkinsTemp-fa13504e-7a22-428a-9e82-84c39cbde4e9/Jenkinsfile - file(s) not in client view.

          at com.perforce.p4java.impl.mapbased.server.cmd.ResultMapParser.handleWarningStr(ResultMapParser.java:164)
          at com.perforce.p4java.impl.mapbased.rpc.OneShotServerImpl.execStreamCmd(OneShotServerImpl.java:545)
          at com.perforce.p4java.impl.mapbased.rpc.OneShotServerImpl.execStreamCmd(OneShotServerImpl.java:461)
          at com.perforce.p4java.impl.mapbased.server.cmd.BaseDelegator.execStreamCmd(BaseDelegator.java:163)
          at com.perforce.p4java.impl.mapbased.server.cmd.PrintDelegator.getFileContents(PrintDelegator.java:56)
          at com.perforce.p4java.impl.mapbased.server.Server.getFileContents(Server.java:2288)
          at org.jenkinsci.plugins.p4.scm.P4SCMFile.content(P4SCMFile.java:145)
          Caused: java.io.IOException
          at org.jenkinsci.plugins.p4.scm.P4SCMFile.content(P4SCMFile.java:147)
          at jenkins.scm.api.SCMFile.contentAsString(SCMFile.java:338)
          at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:104)
          at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:298)
          at hudson.model.ResourceController.execute(ResourceController.java:97)
          at hudson.model.Executor.run(Executor.java:429)
          Checking out p4-buildfarm-//test/jenkins/DSP/Jenkinsfile //${P4_CLIENT}/Jenkinsfile
          //test/jenkins/DSP/... //${P4_CLIENT}/...null into /var/lib/jenkins/workspace/build_DSP-3XHSNXIDZZ35NUQ6WQN4R5QHUQRJU23ZE3YJGA22D46TEDSF2CDQ@script to read Jenkinsfile... p4 client o jenkins-master-build-DSP-DSP-1 +
          ... p4 info +
          ... p4 client o jenkins-master-build-DSP-DSP-1 +
          ... p4 client -i +
          ... client: jenkins-master-build-DSP-DSP-1... p4 client -o jenkins-master-build-DSP-DSP-1 +
          ... p4 info +
          ... p4 counter change +
          ... p4 changes m1 -ssubmitted //jenkins-master-build-DSP-DSP-1/... +
          ... p4 repos -C +
          Building on Node: master... p4 client o jenkins-master-build-DSP-DSP-1 +
          ... p4 info +
          P4 Task: establishing connection.
          ... server: 192.168.56.47:1666
          ... node: jenkins

          P4 Task: syncing files at change: 12057... p4 sync -p /var/lib/jenkins/workspace/build_DSP-3XHSNXIDZZ35NUQ6WQN4R5QHUQRJU23ZE3Y___ +
          duration: (60ms)

          P4 Task: saving built changes.
          Found last change 12056 on syncID jenkins-NODE_NAME-build-DSP-DSP-EXECUTOR_NUMBER... p4 client o jenkins-master-build-DSP-DSP-1 +
          ... p4 info +
          ... p4 changes m20 //jenkins-master-build-DSP-DSP-1/...@12056,12057 +
          ... p4 changes -l -m1 @12057 +
          ... p4 user -o kpark +
          ... p4 describe -s -m51 12057 +
          ... p4 fixes -c12057 +
          ... done

          Running in Durability level: MAX_SURVIVABILITY[Pipeline] nodeRunning on Jenkins
          in /var/lib/jenkins/workspace/build_DSP-3XHSNXIDZZ35NUQ6WQN4R5QHUQRJU23ZE3YJGA22D46TEDSF2CDQ[Pipeline] {[Pipeline] stage[Pipeline]

          { (Sync)[Pipeline] checkout... p4 client -o jenkins-master-build-DSP-DSP-0 + ... p4 info + ... p4 client -o jenkins-master-build-DSP-DSP-0 + ... p4 client -i + ... client: jenkins-master-build-DSP-DSP-0... p4 client -o jenkins-master-build-DSP-DSP-0 + ... p4 info + ... p4 counter change + ... p4 changes -m1 -ssubmitted //jenkins-master-build-DSP-DSP-0/... + P4: no revisions under //jenkins-master-build-DSP-DSP-0/... using latest change: 12057... p4 repos -C + Found last change 12056 on syncID jenkins-NODE_NAME-build-DSP-DSP-EXECUTOR_NUMBER... p4 client -o jenkins-master-build-DSP-DSP-0 + ... p4 info + P4 Task: establishing connection. ... server: 192.168.56.47:1666 ... node: jenkins P4: Polling with range: 12056,now... p4 changes -m20 //jenkins-master-build-DSP-DSP-0/...@12056,now + ... p4 repos -C + Building on Node: master... p4 client -o jenkins-master-build-DSP-DSP-0 + ... p4 info + P4 Task: establishing connection. ... server: 192.168.56.47:1666 ... node: jenkins P4 Task: syncing files at change: 12057... p4 sync -p /var/lib/jenkins/workspace/build_DSP-3XHSNXIDZZ35NUQ6WQN4R5QHUQRJU23ZE3Y___ + duration: (10ms) P4 Task: saving built changes. Found last change 12056 on syncID jenkins-NODE_NAME-build-DSP-DSP-EXECUTOR_NUMBER... p4 client -o jenkins-master-build-DSP-DSP-0 + ... p4 info + ... p4 changes -m20 //jenkins-master-build-DSP-DSP-0/...@12056,12057 + ... p4 client -o jenkins-master-build-DSP-DSP-0 + ... p4 info + ... done[Pipeline] }

          [Pipeline] // stage[Pipeline] stage[Pipeline]

          { (Verify_File)[Pipeline] sh[build_DSP-3XHSNXIDZZ35NUQ6WQN4R5QHUQRJU23ZE3YJGA22D46TEDSF2CDQ] Running shell script + ls -la test total 20 drwxr-xr-x 2 jenkins jenkins 4096 Jun 23 07:17 . drwxr-xr-x 4 jenkins jenkins 4096 Jun 23 07:17 .. -r--r--r-- 1 jenkins jenkins 5 Jun 23 07:17 a.txt -r--r--r-- 1 jenkins jenkins 5 Jun 23 07:17 b.txt -r--r--r-- 1 jenkins jenkins 5 Jun 23 07:17 c.txt[Pipeline] }

          [Pipeline] // stage[Pipeline] stage[Pipeline]

          { (Verify_Content)[Pipeline] sh[build_DSP-3XHSNXIDZZ35NUQ6WQN4R5QHUQRJU23ZE3YJGA22D46TEDSF2CDQ] Running shell script + cat test/a.txt test/b.txt test/c.txt test test test[Pipeline] sh[build_DSP-3XHSNXIDZZ35NUQ6WQN4R5QHUQRJU23ZE3YJGA22D46TEDSF2CDQ] Running shell script + echo cleanup true cleanup true[Pipeline] }

          [Pipeline] // stage[Pipeline] stage[Pipeline]

          { (Declarative: Post Actions)[Pipeline] cleanup... p4 client -o jenkins-master-build-DSP-DSP-0 + ... p4 info + ... p4 client -o jenkins-master-build-DSP-DSP-0 + ... p4 client -i + ... client: jenkins-master-build-DSP-DSP-0... p4 client -o jenkins-master-build-DSP-DSP-0 + ... p4 info + P4 Task: establishing connection. ... server: 192.168.56.47:1666 ... node: jenkins P4 Task: cleanup client: jenkins-master-build-DSP-DSP-0... p4 client -o jenkins-master-build-DSP-DSP-0 - p4 client -o jenkins-master-build-DSP-DSP-0 ... p4 revert /var/lib/jenkins/workspace/build_DSP-3XHSNXIDZZ35NUQ6WQN4R5QHUQRJU23ZE3YJ___ - p4 revert /var/lib/jenkins/workspace/build_DSP-3XHSNXIDZZ35NUQ6WQN4R5QHUQRJU23ZE3YJGA22D46TEDSF2CDQ/... *Replica does not support this command.* [Pipeline] }

          [Pipeline] // stage[Pipeline] }[Pipeline] // node[Pipeline] End of PipelineFinished: SUCCESS

          Kevin Park added a comment - I am sorry that I have missed mentioning buildfarm replica support in Multibranch Pipeline project.   With the following multibranch pipeline project configuration, and the following syntax in Jenkinsfile; [configuration]   [Jenkinsfile] populate: syncOnly( force: true, have: false, modtime: false, parallel: [enable: false, minbytes: '1024', minfiles: '1', threads: '4'] , pin: '', quiet: false, revert: false),   'p4 revert' is still running. And, 'p4 client -d' is not running, which is running when connected to an edge server. P4 Task: remove client: jenkins-master-edge_multi-kpark_bui-kpark_bui-0 ... p4 client -d jenkins-master-edge_multi-kpark_bui-kpark_bui-0 -   Also there is "ERROR: Could not do lightweight checkout, falling back to heavyweight" error as well.   Full Jenkins log is as below.  Console Output Started by user jenkins ... p4 dirs //test/jenkins/* + ... p4 changes -m1 //test/jenkins/DSP/... + ERROR: Could not do lightweight checkout, falling back to heavyweight com.perforce.p4java.exception.RequestException: //jenkinsTemp-fa13504e-7a22-428a-9e82-84c39cbde4e9/Jenkinsfile - file(s) not in client view. at com.perforce.p4java.impl.mapbased.server.cmd.ResultMapParser.handleWarningStr(ResultMapParser.java:164) at com.perforce.p4java.impl.mapbased.rpc.OneShotServerImpl.execStreamCmd(OneShotServerImpl.java:545) at com.perforce.p4java.impl.mapbased.rpc.OneShotServerImpl.execStreamCmd(OneShotServerImpl.java:461) at com.perforce.p4java.impl.mapbased.server.cmd.BaseDelegator.execStreamCmd(BaseDelegator.java:163) at com.perforce.p4java.impl.mapbased.server.cmd.PrintDelegator.getFileContents(PrintDelegator.java:56) at com.perforce.p4java.impl.mapbased.server.Server.getFileContents(Server.java:2288) at org.jenkinsci.plugins.p4.scm.P4SCMFile.content(P4SCMFile.java:145) Caused: java.io.IOException at org.jenkinsci.plugins.p4.scm.P4SCMFile.content(P4SCMFile.java:147) at jenkins.scm.api.SCMFile.contentAsString(SCMFile.java:338) at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:104) at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:298) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) Checking out p4-buildfarm-//test/jenkins/DSP/Jenkinsfile //${P4_CLIENT}/Jenkinsfile //test/jenkins/DSP/... //${P4_CLIENT}/...null into /var/lib/jenkins/workspace/build_DSP-3XHSNXIDZZ35NUQ6WQN4R5QHUQRJU23ZE3YJGA22D46TEDSF2CDQ@script to read Jenkinsfile... p4 client o jenkins-master-build-DSP-DSP -1 + ... p4 info + ... p4 client o jenkins-master-build-DSP-DSP -1 + ... p4 client -i + ... client: jenkins-master-build-DSP-DSP- 1... p4 client -o jenkins-master-build-DSP-DSP -1 + ... p4 info + ... p4 counter change + ... p4 changes m1 -ssubmitted //jenkins-master-build-DSP-DSP -1/... + ... p4 repos -C + Building on Node: master... p4 client o jenkins-master-build-DSP-DSP -1 + ... p4 info + P4 Task: establishing connection. ... server: 192.168.56.47:1666 ... node: jenkins P4 Task: syncing files at change: 12057... p4 sync -p /var/lib/jenkins/workspace/build_DSP-3XHSNXIDZZ35NUQ6WQN4R5QHUQRJU23ZE3Y___ + duration: (60ms) P4 Task: saving built changes. Found last change 12056 on syncID jenkins-NODE_NAME-build-DSP-DSP-EXECUTOR_NUMBER... p4 client o jenkins-master-build-DSP-DSP -1 + ... p4 info + ... p4 changes m20 //jenkins-master-build-DSP-DSP -1/...@12056,12057 + ... p4 changes -l -m1 @12057 + ... p4 user -o kpark + ... p4 describe -s -m51 12057 + ... p4 fixes -c12057 + ... done Running in Durability level: MAX_SURVIVABILITY [Pipeline] nodeRunning on Jenkins in /var/lib/jenkins/workspace/build_DSP-3XHSNXIDZZ35NUQ6WQN4R5QHUQRJU23ZE3YJGA22D46TEDSF2CDQ [Pipeline] { [Pipeline] stage [Pipeline] { (Sync)[Pipeline] checkout... p4 client -o jenkins-master-build-DSP-DSP-0 + ... p4 info + ... p4 client -o jenkins-master-build-DSP-DSP-0 + ... p4 client -i + ... client: jenkins-master-build-DSP-DSP-0... p4 client -o jenkins-master-build-DSP-DSP-0 + ... p4 info + ... p4 counter change + ... p4 changes -m1 -ssubmitted //jenkins-master-build-DSP-DSP-0/... + P4: no revisions under //jenkins-master-build-DSP-DSP-0/... using latest change: 12057... p4 repos -C + Found last change 12056 on syncID jenkins-NODE_NAME-build-DSP-DSP-EXECUTOR_NUMBER... p4 client -o jenkins-master-build-DSP-DSP-0 + ... p4 info + P4 Task: establishing connection. ... server: 192.168.56.47:1666 ... node: jenkins P4: Polling with range: 12056,now... p4 changes -m20 //jenkins-master-build-DSP-DSP-0/...@12056,now + ... p4 repos -C + Building on Node: master... p4 client -o jenkins-master-build-DSP-DSP-0 + ... p4 info + P4 Task: establishing connection. ... server: 192.168.56.47:1666 ... node: jenkins P4 Task: syncing files at change: 12057... p4 sync -p /var/lib/jenkins/workspace/build_DSP-3XHSNXIDZZ35NUQ6WQN4R5QHUQRJU23ZE3Y___ + duration: (10ms) P4 Task: saving built changes. Found last change 12056 on syncID jenkins-NODE_NAME-build-DSP-DSP-EXECUTOR_NUMBER... p4 client -o jenkins-master-build-DSP-DSP-0 + ... p4 info + ... p4 changes -m20 //jenkins-master-build-DSP-DSP-0/...@12056,12057 + ... p4 client -o jenkins-master-build-DSP-DSP-0 + ... p4 info + ... done[Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (Verify_File)[Pipeline] sh[build_DSP-3XHSNXIDZZ35NUQ6WQN4R5QHUQRJU23ZE3YJGA22D46TEDSF2CDQ] Running shell script + ls -la test total 20 drwxr-xr-x 2 jenkins jenkins 4096 Jun 23 07:17 . drwxr-xr-x 4 jenkins jenkins 4096 Jun 23 07:17 .. -r--r--r-- 1 jenkins jenkins 5 Jun 23 07:17 a.txt -r--r--r-- 1 jenkins jenkins 5 Jun 23 07:17 b.txt -r--r--r-- 1 jenkins jenkins 5 Jun 23 07:17 c.txt[Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (Verify_Content)[Pipeline] sh[build_DSP-3XHSNXIDZZ35NUQ6WQN4R5QHUQRJU23ZE3YJGA22D46TEDSF2CDQ] Running shell script + cat test/a.txt test/b.txt test/c.txt test test test[Pipeline] sh[build_DSP-3XHSNXIDZZ35NUQ6WQN4R5QHUQRJU23ZE3YJGA22D46TEDSF2CDQ] Running shell script + echo cleanup true cleanup true[Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (Declarative: Post Actions)[Pipeline] cleanup... p4 client -o jenkins-master-build-DSP-DSP-0 + ... p4 info + ... p4 client -o jenkins-master-build-DSP-DSP-0 + ... p4 client -i + ... client: jenkins-master-build-DSP-DSP-0... p4 client -o jenkins-master-build-DSP-DSP-0 + ... p4 info + P4 Task: establishing connection. ... server: 192.168.56.47:1666 ... node: jenkins P4 Task: cleanup client: jenkins-master-build-DSP-DSP-0... p4 client -o jenkins-master-build-DSP-DSP-0 - p4 client -o jenkins-master-build-DSP-DSP-0 ... p4 revert /var/lib/jenkins/workspace/build_DSP-3XHSNXIDZZ35NUQ6WQN4R5QHUQRJU23ZE3YJ___ - p4 revert /var/lib/jenkins/workspace/build_DSP-3XHSNXIDZZ35NUQ6WQN4R5QHUQRJU23ZE3YJGA22D46TEDSF2CDQ/... *Replica does not support this command.* [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // node [Pipeline] End of PipelineFinished: SUCCESS

            Unassigned Unassigned
            s3037560 Kevin Park
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: