Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-56248

P4Plugin Polls Workspaces With Disabled Polling (poll: false)

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • p4-plugin
    • Jenkins: 2.150.3 (official docker image)
      P4 Plugin: 1.9.6

      Hello

      P4Plugin polls all workspaces used in job even if poll parameter is disabled in checkout.

      Steps to reproduce

      1. Create pipeline job test4 (see below)
      2. Start pipeline job test4
      3. See "Perforce Software Polling Log" and Perforce Server Log

       

       

      pipeline {
          agent(none)
       
          options {
              ansiColor("xterm")
              buildDiscarder(logRotator(numToKeepStr: "10"))
              disableConcurrentBuilds()
              disableResume()
              skipDefaultCheckout(true)
              timestamps()
          }
       
          triggers {
              pollSCM("* * * * *")
          }
       
          stages {
              stage("Master") {
                  agent {
                      label("master")
                  }
       
                  steps {
                      checkout perforce(credential: 'perforce-ticket-jenkins', populate: flushOnly(pin: '', quiet: false),
                              workspace: streamSpec(charset: 'none', format: 'test4-master', pinHost: true, streamName: '//playground/jenkins-main'))
                  }
              }
       
              stage("Win-1") {
                  agent {
                      label("k02-bld-win-01")
                  }
       
                  steps {
                      checkout changelog: false, poll: false, scm: perforce(credential: 'perforce-ticket-jenkins', populate: flushOnly(pin: '', quiet: false),
                              workspace: streamSpec(charset: 'none', format: 'test4-win-1', pinHost: true, streamName: '//playground/jenkins-main'))
                  }
              }
      
              stage("Win-2") {
                  agent {
                      label("k02-bld-win-02")
                  }
       
                  steps {
                      checkout changelog: false, poll: false, scm: perforce(credential: 'perforce-ticket-jenkins', populate: flushOnly(pin: '', quiet: false),
                              workspace: streamSpec(charset: 'none', format: 'test4-win-2', pinHost: true, streamName: '//playground/jenkins-main'))
                  }
              }
          }
      }
      

       

      Perforce Software Polling Log

       

      Started on Feb 22, 2019 3:45:00 PM
      P4: Polling on: master with:test4-master
      Found last change 1180191 on syncID test4-master
      ... p4 client -o test4-master +
      ... p4 info +
      ... p4 info +
      ... p4 client -o test4-master +
      ...   No change in client detected.
       
      P4 Task: establishing connection.
      ... server: perforce:1666
      ... node: k02-jenkins
      P4: Polling with range: 1180191,now
      ... p4 changes -m20 //test4-master/...@1180191,now +
      ... p4 repos -C +
      P4: Polling on: master with:test4-win-1
      Found last change 1180191 on syncID test4-win-1
      ... p4 client -o test4-win-1 +
      ... p4 info +
      ... p4 info +
      ... p4 client -o test4-win-1 +
      ...   No change in client detected.
       
      P4 Task: establishing connection.
      ... server: perforce:1666
      ... node: k02-jenkins
      P4: Polling with range: 1180191,now
      ... p4 changes -m20 //test4-win-1/...@1180191,now +
      ... p4 repos -C +
      P4: Polling on: master with:test4-win-2
      Found last change 1180191 on syncID test4-win-2
      ... p4 client -o test4-win-2 +
      ... p4 info +
      ... p4 info +
      ... p4 client -o test4-win-2 +
      ...   No change in client detected.
       
      P4 Task: establishing connection.
      ... server: perforce:1666
      ... node: k02-jenkins
      P4: Polling with range: 1180191,now
      ... p4 changes -m20 //test4-win-2/...@1180191,now +
      ... p4 repos -C +
      Done. Took 0.14 sec
      No changes
      

       

      Perforce Server Error Log

      4,1550834520,532931606,2019/02/22 13:22:00 532931606,210535,1,jenkins,test4-win-1,user-repos,10.10.20.103,jenkins.p4-plugin,1.9.6/Linux,-C,error,6,95,Client %27test4-win-1%27 can only be used from host %27k02-bld-win-01%27.
       
      4,1550834520,580422337,2019/02/22 13:22:00 580422337,210544,1,jenkins,test4-win-2,user-repos,10.10.20.103,jenkins.p4-plugin,1.9.6/Linux,-C,error,6,95,Client %27test4-win-2%27 can only be used from host %27k02-bld-win-02%27.
       
       
      

       

      As you can see jenkins tries to poll all three workspaces: test4-master (poll: true) + test4-win-1 (poll: false) + test4-win-2 (poll: false) from master.

       

      Cause

      WorkflowJob.poll() uses PerforceScm.getKey() to find a polling baseline for workspaces.

      PerforceScm.getKey() returns the same SCM key for different workspaces because it doesn't include a workspace name.

      Result: WorkflowJob.poll() polls all workspaces with the same SCM key.

      PR

      I have created a pull request with a possible fix of this problem:

      https://github.com/jenkinsci/p4-plugin/pull/95

       

      Please fix this bug.

      Thank you

          [JENKINS-56248] P4Plugin Polls Workspaces With Disabled Polling (poll: false)

          Paul Allen added a comment -

          Need to verify. Please test with the latest version of jenkins.

          Paul Allen added a comment - Need to verify. Please test with the latest version of jenkins.

          Jenkins 2.164.3 has the same behavior as 2.150.3.

          Mykola Ulianytskyi added a comment - Jenkins 2.164.3 has the same behavior as 2.150.3.

          Tested pull request https://github.com/jenkinsci/p4-plugin/pull/95 and looks to be working fine with Jenkins version 2.164.3.

          Charusheela Bopardikar added a comment - Tested pull request  https://github.com/jenkinsci/p4-plugin/pull/95  and looks to be working fine with Jenkins version 2.164.3.

          Charusheela Bopardikar added a comment - - edited

          The code change as suggested in the PR will have an impact on other customers. Need to do a thorough analysis before accepting the PR.

          Meanwhile, I tested below workaround as suggested by p4paul and it has exactly the same effect as the PR without any code change.

          Changed the view mapping to make it unique using the name parameter since ${P4_CLIENT} is not expanded when the key is evaluated.

           

          view: '//depot/pipeline_poll_false/sub1/... //jenkins-${NODE_NAME}${JOB_NAME}${EXECUTOR_NUMBER}-sub1/...'

          and

          view: '//depot/pipeline_poll_false/sub1/... //jenkins-${NODE_NAME}${JOB_NAME}${EXECUTOR_NUMBER}-sub2/...'

           So, the script looks like this:

          pipeline {
            agent { label 'master' }
            stages {
              stage("Repro") {
                steps {
                  script {
                     echo "Starting"           
          checkout poll: true, scm: perforce(credential: 'JenkinsMasterAdminUser', populate: autoClean(delete: true, modtime: false, parallel: [enable: false, minbytes: '1024', minfiles: '1', threads: '4'], pin: '', quiet: false, replace: true, tidy: false), workspace: manualSpec(charset: 'none', cleanup: false, name: 'jenkins-${NODE_NAME}-${JOB_NAME}-${EXECUTOR_NUMBER}-sub1', pinHost: false, spec: clientSpec(allwrite: false, backup: true, changeView: '', clobber: true, compress: false, line: 'LOCAL', locked: false, modtime: false, rmdir: false, serverID: '', streamName: '', type: 'WRITABLE', view: '//depot/pipeline_poll_false/sub1/...  //jenkins-${NODE_NAME}-${JOB_NAME}-${EXECUTOR_NUMBER}-sub1/...')))           
          
          checkout poll: false, scm: perforce(credential: 'JenkinsMasterAdminUser', populate: autoClean(delete: true, modtime: false, parallel: [enable: false, minbytes: '1024', minfiles: '1', threads: '4'], pin: '', quiet: false, replace: true, tidy: false), workspace: manualSpec(charset: 'none', cleanup: false, name: 'jenkins-${NODE_NAME}-${JOB_NAME}-${EXECUTOR_NUMBER}-sub2', pinHost: false, spec: clientSpec(allwrite: false, backup: true, changeView: '', clobber: true, compress: false, line: 'LOCAL', locked: false, modtime: false, rmdir: false, serverID: '', streamName: '', type: 'WRITABLE', view: '//depot/pipeline_poll_false/sub2/...  //jenkins-${NODE_NAME}-${JOB_NAME}-${EXECUTOR_NUMBER}-sub2/...')))           sh 'echo WS IS $P4_CLIENT'        }
                }
              }
            }
          }

          Charusheela Bopardikar added a comment - - edited The code change as suggested in the PR will have an impact on other customers. Need to do a thorough analysis before accepting the PR. Meanwhile, I tested below workaround as suggested by p4paul  and it has exactly the same effect as the PR without any code change. Changed the view mapping to make it unique using the name parameter since ${P4_CLIENT} is not expanded when the key is evaluated.   view: '//depot/pipeline_poll_false/sub1/... //jenkins-${NODE_NAME}${JOB_NAME}${EXECUTOR_NUMBER}-sub1/...' and view: '//depot/pipeline_poll_false/sub1/... //jenkins-${NODE_NAME}${JOB_NAME}${EXECUTOR_NUMBER}-sub2/...'  So, the script looks like this: pipeline { agent { label 'master' } stages { stage( "Repro" ) { steps { script { echo "Starting" checkout poll: true, scm: perforce(credential: 'JenkinsMasterAdminUser' , populate: autoClean(delete: true, modtime: false, parallel: [enable: false, minbytes: '1024' , minfiles: '1' , threads: '4' ], pin: '', quiet: false, replace: true, tidy: false), workspace: manualSpec(charset: ' none ', cleanup: false, name: ' jenkins-${NODE_NAME}-${JOB_NAME}-${EXECUTOR_NUMBER}-sub1 ', pinHost: false, spec: clientSpec(allwrite: false, backup: true, changeView: ' ', clobber: true, compress: false, line: ' LOCAL ', locked: false, modtime: false, rmdir: false, serverID: ' ', streamName: ' ', type: ' WRITABLE ', view: ' //depot/pipeline_poll_false/sub1/... //jenkins-${NODE_NAME}-${JOB_NAME}-${EXECUTOR_NUMBER}-sub1/...'))) checkout poll: false, scm: perforce(credential: 'JenkinsMasterAdminUser' , populate: autoClean(delete: true, modtime: false, parallel: [enable: false, minbytes: '1024' , minfiles: '1' , threads: '4' ], pin: '', quiet: false, replace: true, tidy: false), workspace: manualSpec(charset: ' none ', cleanup: false, name: ' jenkins-${NODE_NAME}-${JOB_NAME}-${EXECUTOR_NUMBER}-sub2 ', pinHost: false, spec: clientSpec(allwrite: false, backup: true, changeView: ' ', clobber: true, compress: false, line: ' LOCAL ', locked: false, modtime: false, rmdir: false, serverID: ' ', streamName: ' ', type: ' WRITABLE ', view: ' //depot/pipeline_poll_false/sub2/... //jenkins-${NODE_NAME}-${JOB_NAME}-${EXECUTOR_NUMBER}-sub2/... '))) sh ' echo WS IS $P4_CLIENT' } } } } }

          Meanwhile, I tested below workaround as suggested by p4paul and it has exactly the same effect as the PR without any code change.

          The workaround doesn't work for streamSpec. It can be used only with manualSpec.

          PR fixes the issue with streamSpec.

          Mykola Ulianytskyi added a comment - Meanwhile, I tested below workaround as suggested by  p4paul  and it has exactly the same effect as the PR without any code change. The workaround doesn't work for streamSpec. It can be used only with manualSpec. PR fixes the issue with streamSpec.

          Adam Smith added a comment -

          Thank you for the workaround cbopardikar, this fixed my use case (which is using manualSpec). That said, this has been in backlog for 9 months now – any progress here p4karl?

          Adam Smith added a comment - Thank you for the workaround cbopardikar , this fixed my use case (which is using manualSpec). That said, this has been in backlog for 9 months now – any progress here p4karl ?

          Karl Wirth added a comment - - edited

          Hi asmith_pipeworks - I'm just support sorry.  cbopardikar - Can you take another look at this? Thanks in advance.

          Karl Wirth added a comment - - edited Hi asmith_pipeworks - I'm just support sorry.  cbopardikar - Can you take another look at this? Thanks in advance.

          Paul Allen added a comment -

          TODO: retest this issue after updating plugin dependencies.

          Paul Allen added a comment - TODO: retest this issue after updating plugin dependencies.

          Karl Wirth added a comment -

          FYI - Still fails on 1.11.1.

          Karl Wirth added a comment - FYI - Still fails on 1.11.1.

          Adam Smith added a comment -

          Checking in on this again. cbopardikar your comment 18 months ago indicates that you were going to do a thorough analysis on impact to other customers, but doesn't appear to have any follow up. In the meantime, this customer is being impacted right now by the defect.

          Does Perforce have any plan to either accept the submitted PR or propose a fix of their own? p4karl any insight from support?

          Thanks all.

          Adam Smith added a comment - Checking in on this again. cbopardikar your comment 18 months ago indicates that you were going to do a thorough analysis on impact to other customers, but doesn't appear to have any follow up. In the meantime, this customer is being impacted right now by the defect. Does Perforce have any plan to either accept the submitted PR or propose a fix of their own? p4karl any insight from support? Thanks all.

            cbopardikar Charusheela Bopardikar
            lystor Mykola Ulianytskyi
            Votes:
            4 Vote for this issue
            Watchers:
            12 Start watching this issue

              Created:
              Updated: