Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-37462

Polling breaks with multiple slaves with own workspace

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Critical Critical
    • p4-plugin
    • None
    • Jenkins ver 2.17, p4 plugin 1.4.4

      When using multiple slaves, each with their own perforce workspace, polling appears to be broken causing endless rebuilds.

      One slave does the polling, finding changes:

      P4 Task: establishing connection.
      ... server: perforce.somewhere.com:1666
      ... node: A
      P4: Polling with cstat: //

      {some-path}/...
      ... p4 cstat //{some-path}

      /... +
      ... p4 change -o 113712 +
      ... found change: 113712
      ... p4 change -o 113686 +
      ... found change: 113686

      Triggering a build on another host:

      P4 Task: establishing connection.
      ... server: perforce.somewhere.com:1666
      ... node: B

      Which already has all the changes!

      It seems that the p4-plugin is relying only on the perforce servers knowledge of what the client has to decide if a new build is needed, which breaks if multiple clients are polling/building.

      Maybe it would be better to save the last changelist id on the master and only build if a new changelist is found on when querying the server?

          [JENKINS-37462] Polling breaks with multiple slaves with own workspace

          Nils Carlson created issue -
          Tim Fredrickson made changes -
          Link New: This issue is related to JENKINS-34037 [ JENKINS-34037 ]

          Would be nice to get some eyes on this

          Tim Fredrickson added a comment - Would be nice to get some eyes on this
          Nils Carlson made changes -
          Assignee New: Paul Allen [ p4paul ]

          Nils Carlson added a comment -

          Hi,

          Any chance of getting eyes on this? This is really impairing our use of jenkins with perforce.

          Nils Carlson added a comment - Hi, Any chance of getting eyes on this? This is really impairing our use of jenkins with perforce.

          Paul Allen added a comment -

          Polling with cstat is a fall back situation if the P4 plugin was not able to find the last built change. Are you throwing the Perforce workspace away each time? or sharing the Perforce workspace between slaves?

          Paul Allen added a comment - Polling with cstat is a fall back situation if the P4 plugin was not able to find the last built change. Are you throwing the Perforce workspace away each time? or sharing the Perforce workspace between slaves?

          Nils Carlson added a comment -

          Hi,

          I'm pretty sure we have a dedicated workspace per slave.
          I am running a pipeline job, with the following setup:

          master polls on the Jenkinsfile in own workspace on node A
          slaves build inside the pipeline and do their own polling on node B and C. Node C has been disabled, but I still get unwanted polling.

          But looking at the output of the perforce plugin in detail I do see strange things:

          ------------------------

          [Pipeline] stage (Build)
          Using the ‘stage’ step without a block argument is deprecated
          Entering stage Build
          Proceeding
          [Pipeline] echo
          Bulding <snip>
          [Pipeline] node
          Still waiting to schedule task
          Waiting for next available executor on docker
          Running on nodeB in <snip>
          [Pipeline] {
          [Pipeline] echo

          perforce depot path: "<snip>"
          perforce view: "jenkins-<job>-nodeB"
          perforce view spec: "<snip>"
          perforce populate type: "AutoCleanImpl"
          [Pipeline] checkout
          ... p4 client o jenkins<job>-NodeB+
          ... p4 info +

          P4 Task: establishing connection.
          ... server: <snip>
          ... node: nodeA
          ... p4 client o jenkins<job>-nodeB +
          ... p4 client -i +
          ... client: jenkins-<job>-nodeB
          ... p4 client o jenkins<job>-nodeB +
          ... p4 info +
          ... p4 counter change +
          ... p4 changes m1 -ssubmitted //jenkins<job>-nodeB/... +
          Building on Node: master
          ... p4 client o jenkins<job>-nodeB +
          ... p4 info +

          P4 Task: establishing connection.
          ... server: <snip>
          ... node: nodeB

          P4 Task: reverting all pending and shelved revisions.
          ... p4 revert <snip> +
          ... rm [abandoned files]
          duration: (3ms)

          P4 Task: skipping clean, no options set.
          P4 Task: syncing files at change: <snip>
          ... p4 sync <snip> +
          ... p4 client -i +
          duration: 0m 2s

          P4 Task: saving built changes.
          ... p4 client o jenkins<job>-nodeB +
          ... p4 info +
          ... p4 changes m100 //jenkins<job>-nodeB/...<snip> +

          ------------------------------------------------

          Notice that for the same workspace belonging to nodeB the master node, nodeA, establishes a connection during the pipeline.
          Could this be causing problems somehow? Is this wanted behavior?

          Nils Carlson added a comment - Hi, I'm pretty sure we have a dedicated workspace per slave. I am running a pipeline job, with the following setup: master polls on the Jenkinsfile in own workspace on node A slaves build inside the pipeline and do their own polling on node B and C. Node C has been disabled, but I still get unwanted polling. But looking at the output of the perforce plugin in detail I do see strange things: ------------------------ [Pipeline] stage (Build) Using the ‘stage’ step without a block argument is deprecated Entering stage Build Proceeding [Pipeline] echo Bulding <snip> [Pipeline] node Still waiting to schedule task Waiting for next available executor on docker Running on nodeB in <snip> [Pipeline] { [Pipeline] echo perforce depot path: "<snip>" perforce view: "jenkins-<job>-nodeB" perforce view spec: "<snip>" perforce populate type: "AutoCleanImpl" [Pipeline] checkout ... p4 client o jenkins <job>-NodeB+ ... p4 info + P4 Task: establishing connection. ... server: <snip> ... node: nodeA ... p4 client o jenkins <job>-nodeB + ... p4 client -i + ... client: jenkins-<job>-nodeB ... p4 client o jenkins <job>-nodeB + ... p4 info + ... p4 counter change + ... p4 changes m1 -ssubmitted //jenkins <job>-nodeB/... + Building on Node: master ... p4 client o jenkins <job>-nodeB + ... p4 info + P4 Task: establishing connection. ... server: <snip> ... node: nodeB P4 Task: reverting all pending and shelved revisions. ... p4 revert <snip> + ... rm [abandoned files] duration: (3ms) P4 Task: skipping clean, no options set. P4 Task: syncing files at change: <snip> ... p4 sync <snip> + ... p4 client -i + duration: 0m 2s P4 Task: saving built changes. ... p4 client o jenkins <job>-nodeB + ... p4 info + ... p4 changes m100 //jenkins <job>-nodeB/...<snip> + ------------------------------------------------ Notice that for the same workspace belonging to nodeB the master node, nodeA, establishes a connection during the pipeline. Could this be causing problems somehow? Is this wanted behavior?

          Paul Allen added a comment -

          Please can you provide the pipeline script. I'd like to know which DSLs you are using to sync the files (checkout scm, p4sync, checkout).

          Paul Allen added a comment - Please can you provide the pipeline script. I'd like to know which DSLs you are using to sync the files (checkout scm, p4sync, checkout).

          Nils Carlson added a comment -

          This is the function I use:

          def perforceCheckout(def depotPathStr, def viewName, def forceClean=false) {
          def cleanType = 'AutoCleanImpl'
          if (forceClean)

          { cleanType = 'ForceCleanImpl' }

          p4sync(credential: "p4-read-only",
          depotPath: depotPathStr,
          poll: true,
          populate: [$class: cleanType ])
          }

          And this is the XML from the defining the Jenkinsfile step for the pipeline:

          <definition class="org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition">
          <scm class="org.jenkinsci.plugins.p4.PerforceScm">
          <credential>p4-read-only</credential>
          <workspace class="org.jenkinsci.plugins.p4.workspace.ManualWorkspaceImpl">
          <name>job</name>
          <spec>
          <view>....view....</view>
          </spec>
          </workspace>
          <filter>
          <org.jenkinsci.plugins.p4.filters.FilterPathImpl>
          <path>//</path>
          </org.jenkinsci.plugins.p4.filters.FilterPathImpl>
          </filter>
          <populate class="org.jenkinsci.plugins.p4.populate.ForceCleanImpl"/>
          </scm>
          <scriptPath>Jenkinsfile</scriptPath>
          </definition>

          Changes in the pipeline shouldn't trigger any jobs, therefore the filter.

          Nils Carlson added a comment - This is the function I use: def perforceCheckout(def depotPathStr, def viewName, def forceClean=false) { def cleanType = 'AutoCleanImpl' if (forceClean) { cleanType = 'ForceCleanImpl' } p4sync(credential: "p4-read-only", depotPath: depotPathStr, poll: true, populate: [$class: cleanType ] ) } And this is the XML from the defining the Jenkinsfile step for the pipeline: <definition class="org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition"> <scm class="org.jenkinsci.plugins.p4.PerforceScm"> <credential>p4-read-only</credential> <workspace class="org.jenkinsci.plugins.p4.workspace.ManualWorkspaceImpl"> <name>job</name> <spec> <view>....view....</view> </spec> </workspace> <filter> <org.jenkinsci.plugins.p4.filters.FilterPathImpl> <path>//</path> </org.jenkinsci.plugins.p4.filters.FilterPathImpl> </filter> <populate class="org.jenkinsci.plugins.p4.populate.ForceCleanImpl"/> </scm> <scriptPath>Jenkinsfile</scriptPath> </definition> Changes in the pipeline shouldn't trigger any jobs, therefore the filter.

          Nils Carlson added a comment -

          I saw this filter now: https://issues.jenkins-ci.org/browse/JENKINS-32814 letting one only poll on the master using the lastBuild. This seems to be what the git plugin is doing normally, would it be possible to do this from a pipeline build? Are there any downsides to this?

          Nils Carlson added a comment - I saw this filter now: https://issues.jenkins-ci.org/browse/JENKINS-32814 letting one only poll on the master using the lastBuild. This seems to be what the git plugin is doing normally, would it be possible to do this from a pipeline build? Are there any downsides to this?

            p4paul Paul Allen
            pyssling Nils Carlson
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: