Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-47427

Poll per change with pipeline jobs

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      Add ability to poll per change when Jenkinsfile includes a checkout with poll per change ('filter: [ incremental(true) ]'). For example:
                      checkout perforce(credential: 'myCredential',
                               filter: [ incremental(true) ],
                               populate: forceClean(have: true,
                                       parallel:
                                              [
                                               enable: false,
                                               minbytes: '1024',
       

      At the moment when the pipeline polls it will build all changelists within the view since last poll.

        Attachments

          Issue Links

            Activity

            Hide
            p4karl Karl Wirth added a comment -

            Limited behavior is documented in new job.

            Show
            p4karl Karl Wirth added a comment - Limited behavior is documented in new job.
            Hide
            p4karl Karl Wirth added a comment -

            Poll per change is tricky when you have one or more p4 sync commands involved. An extreme example is if you have two p4sync commands that sync from different P4D servers in the same Jenkinsfile. In this case advanced logic is needed for the polling to decide which one triggers polling and Jenkins may not even support this scenario.

            For now Poll per change only fully works (trigger a job per each submitted changelist and sync up to that changelist only) in the limited scenario documented in JENKINS-51525.

            Show
            p4karl Karl Wirth added a comment - Poll per change is tricky when you have one or more p4 sync commands involved. An extreme example is if you have two p4sync commands that sync from different P4D servers in the same Jenkinsfile. In this case advanced logic is needed for the polling to decide which one triggers polling and Jenkins may not even support this scenario. For now Poll per change only fully works (trigger a job per each submitted changelist and sync up to that changelist only) in the limited scenario documented in JENKINS-51525 .
            Hide
            p4karl Karl Wirth added a comment -

            This defect is being closed in favor of JENKINS-51525.

            The basic poll per change logic exists and works with pipeline.

            Additional investigation and work is needed to get it to work in the more difficult to predetermine scenarios where you rely on the Jenkinsfile contents to inform the polling mechanism on what paths need to be polled and need to cope with situations when the paths in the Jenkinsfile have changed.

            Show
            p4karl Karl Wirth added a comment - This defect is being closed in favor of JENKINS-51525 . The basic poll per change logic exists and works with pipeline. Additional investigation and work is needed to get it to work in the more difficult to predetermine scenarios where you rely on the Jenkinsfile contents to inform the polling mechanism on what paths need to be polled and need to cope with situations when the paths in the Jenkinsfile have changed.
            Hide
            p4karl Karl Wirth added a comment -

            Just updating this bug with further information on current behavior for reference only.

             

            With multibranch poll per change doesnt poll every change but does build only the next change that needs to be built.For example:

            (1) Add CL 626, 627 and 628.

            (2) The multi branch top level poll is configured for 1 min (see below). When it triggers it detects that 625 to 628 exist.

            (3) The job checks SyncID and correctly finds '625' as the previous last sync.          __

            Found last change 625 on syncID jenkins-NODE_NAME-MultiBranch_PollPerChange-main-EXECUTOR_NUMBER

            (4) Due to filtering only the next changelist is synced:

                      P4 Task: syncing files at change: 626

            (5) Jenkins records the last sucseful buils for this branch correctly:

                      branches/main/lastSuccessful/build.xml:          <change>626</change>

            (6) But (and I'm guessing this is the cause) sets '628' in the following file:

                      branches/main/scm-revision-hash.xml

             The next time a poll runs it will only fire if a new changelist has been created but will build changelist '627' only (so will now be building 2 changelists behind current).

             

            My Jenkinsfile:

                 1	pipeline {
                 2	  agent { label 'master' }
                 3	  options { skipDefaultCheckout() }
                 4	  stages {
                 5	    stage("Repro") {
                 6	      steps {
                 7	        script {
                 8	           checkout perforce(credential: 'MasterServer', filter: [incremental(true)], populate: autoClean(delete: true, modtime: false, parallel: [enable: false, minbytes: '1024', minfiles: '1', threads: '4'], pin: '', quiet: true, replace: true, tidy: false), workspace: manualSpec(charset: 'none', name: 'jenkins-${NODE_NAME}-${JOB_NAME}-${EXECUTOR_NUMBER}', pinHost: false, spec: clientSpec(allwrite: false, backup: false, clobber: true, compress: false, line: 'LOCAL', locked: false, modtime: false, rmdir: false, serverID: '', streamName: '', type: 'WRITABLE', view: '//depot/multibranch_poll_per_change/main/... //jenkins-${NODE_NAME}-${JOB_NAME}-${EXECUTOR_NUMBER}/...')))
                 9	           sh "pwd"
                10	           sh "ls -l"
                11		   sh "cat -n *"
                12		   sh "echo ${env.BRANCH_NAME}"
                13	        }
                14	      }
                15	    }
                16	  }
                17	}
            

             

            Show
            p4karl Karl Wirth added a comment - Just updating this bug with further information on current behavior for reference only.   With multibranch poll per change doesnt poll every change but does build only the next change that needs to be built.For example: (1) Add CL 626, 627 and 628. (2) The multi branch top level poll is configured for 1 min (see below). When it triggers it detects that 625 to 628 exist. (3) The job checks SyncID and correctly finds '625' as the previous last sync.          __ Found last change 625 on syncID jenkins-NODE_NAME-MultiBranch_PollPerChange-main-EXECUTOR_NUMBER (4) Due to filtering only the next changelist is synced:           P4 Task: syncing files at change: 626 (5) Jenkins records the last sucseful buils for this branch correctly:           branches/main/lastSuccessful/build.xml:          <change>626</change> (6) But (and I'm guessing this is the cause) sets '628' in the following file:           branches/main/scm-revision-hash.xml  The next time a poll runs it will only fire if a new changelist has been created but will build changelist '627' only (so will now be building 2 changelists behind current).   My Jenkinsfile: 1 pipeline { 2 agent { label 'master' } 3 options { skipDefaultCheckout() } 4 stages { 5 stage( "Repro" ) { 6 steps { 7 script { 8 checkout perforce(credential: 'MasterServer' , filter: [incremental( true )], populate: autoClean(delete: true , modtime: false , parallel: [enable: false , minbytes: '1024' , minfiles: '1' , threads: '4' ], pin: '', quiet: true , replace: true , tidy: false ), workspace: manualSpec(charset: ' none ', name: ' jenkins-${NODE_NAME}-${JOB_NAME}-${EXECUTOR_NUMBER} ', pinHost: false , spec: clientSpec(allwrite: false , backup: false , clobber: true , compress: false , line: ' LOCAL ', locked: false , modtime: false , rmdir: false , serverID: ' ', streamName: ' ', type: ' WRITABLE ', view: ' //depot/multibranch_poll_per_change/main/... //jenkins-${NODE_NAME}-${JOB_NAME}-${EXECUTOR_NUMBER}/...'))) 9 sh "pwd" 10 sh "ls -l" 11 sh "cat -n *" 12 sh "echo ${env.BRANCH_NAME}" 13 } 14 } 15 } 16 } 17 }  
            Hide
            p4karl Karl Wirth added a comment -

            Based on last findings have created a seperate feature request to implement poll per change for multibranch pipeline.

            JENKINS-52066

            Show
            p4karl Karl Wirth added a comment - Based on last findings have created a seperate feature request to implement poll per change for multibranch pipeline. JENKINS-52066

              People

              Assignee:
              p4paul Paul Allen
              Reporter:
              p4karl Karl Wirth
              Votes:
              2 Vote for this issue
              Watchers:
              7 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: