Minor Feature request: Implement poll per change on multibranch pipeline jobs.
With multibranch poll per change doesnt poll every change but does build only the next change that needs to be built.For example:
(1) Add CL 626, 627 and 628.
(2) The multi branch top level poll is configured for 1 min (see below). When it triggers it detects that 625 to 628 exist.
(3) The job checks SyncID and correctly finds '625' as the previous last sync.
Found last change 625 on syncID jenkins-NODE_NAME-MultiBranch_PollPerChange-main-EXECUTOR_NUMBER
(4) Due to filtering only the next changelist is synced:
P4 Task: syncing files at change: 626
(5) Jenkins records the last successful build for this branch correctly:
branches/main/lastSuccessful/build.xml: <change>626</change>
(6) But (and I'm guessing this is the cause) sets '628' in the following file:
branches/main/scm-revision-hash.xml
The next time a poll runs it will only fire if a new changelist has been created but will build changelist '627' only (so will now be building 2 changelists behind current).
My Jenkinsfile:
1 pipeline {
2 agent { label 'master' }
3 options { skipDefaultCheckout() }
4 stages {
5 stage("Repro") {
6 steps {
7 script {
8 checkout perforce(credential: 'MasterServer', filter: [incremental(true)], populate: autoClean(delete: true, modtime: false, parallel: [enable: false, minbytes: '1024', minfiles: '1', threads: '4'], pin: '', quiet: true, replace: true, tidy: false), workspace: manualSpec(charset: 'none', name: 'jenkins-${NODE_NAME}-${JOB_NAME}-${EXECUTOR_NUMBER}', pinHost: false, spec: clientSpec(allwrite: false, backup: false, clobber: true, compress: false, line: 'LOCAL', locked: false, modtime: false, rmdir: false, serverID: '', streamName: '', type: 'WRITABLE', view: '//depot/multibranch_poll_per_change/main/... //jenkins-${NODE_NAME}-${JOB_NAME}-${EXECUTOR_NUMBER}/...')))
9 sh "pwd"
10 sh "ls -l"
11 sh "cat -n *"
12 sh "echo ${env.BRANCH_NAME}"
13 }
14 }
15 }
16 }
17 }
- is related to
-
JENKINS-51525 Pipeline: Poll per change syncs to head when using 'checkout' or 'p4 sync' step.
-
- Closed
-
- relates to
-
-
- Closed
-
-
-
- Closed
-
[JENKINS-52066] Poll per change with multibranch pipeline jobs
As JENKINS-51525 seems to be moving towards a close, I'm looking forward to hearing updates on this ticket too. These two would allow us to actually do incremental builds with explicit checkout, which would be very nice.
Karl Wirth
added a comment - Hi danielresolutiongames. No change in behavior.
Poll per change will cause only the next change to build, but subsequent pipeline scans will only trigger a build for a change submitted after the previous job ?completed?. Therefore you get:
Submit 1,2,3. Pipeline scan builds 1. Next pipeline scan detects no new changes since 3. Submit 4. Next pipeline scan detects new changes (4) and builds 2.
The fix for JENKINS-51525 has not implemented the poll per change against multibranch. I will therefore highlight this to the product managers.
Karl Wirth
added a comment - Hi danielresolutiongames . No change in behavior.
Poll per change will cause only the next change to build, but subsequent pipeline scans will only trigger a build for a change submitted after the previous job ?completed?. Therefore you get:
Submit 1,2,3.
Pipeline scan builds 1.
Next pipeline scan detects no new changes since 3.
Submit 4.
Next pipeline scan detects new changes (4) and builds 2.
The fix for JENKINS-51525 has not implemented the poll per change against multibranch. I will therefore highlight this to the product managers. Hi p4karl. Thanks for the update. It's really unfortunate that this isn't working.
Paul Allen
added a comment - 1.9.6 adds support for MultiBranch Scan per change for 'Helix Branches' implemented as a Polling Filter in the Advanced options. If you disable implicit checkout and use 'p4sync' or 'checkout' steps then you need to add the poll per change filter to the step as well as enabling MultiBranch Scan per change.
Paul Allen
added a comment - 1.9.6 adds support for MultiBranch Scan per change for 'Helix Branches' implemented as a Polling Filter in the Advanced options. If you disable implicit checkout and use 'p4sync' or 'checkout' steps then you need to add the poll per change filter to the step as well as enabling MultiBranch Scan per change. Paul Allen
added a comment - Released in 1.9.6 I'm not seeing this working as intended in 1.9.6, but rather just as broken as before. I have several multibranch pipelines that poll e.g. once per minute, and are still lagging behind on their builds. The scanning log says something like this:
{{
... p4 changes -m1 //project/master/...@now -
p4 changes -m1 //project/master/...@now
Change 41254 on 2019/01/22 by bjorn@bjorn_project_workstation 'RES-11336 Update localization w'
... p4 files -e //project/master/Jenkinsfile -
p4 files -e //project/master/Jenkinsfile
//project/master/Jenkinsfile#3 - edit change 39828 (text)
‘Jenkinsfile’ found
No changes detected: master (still at 41254)
}}
Whereas the last build is only at @41189. When manually triggering a build, the build correctly contains the next changelist, in this example @41224. It does not trigger any further builds automatically, but if I keep pressing "Build now" I keep getting builds until I'm caught up again.
In my eyes this inconsistency is exactly as it was before this bug was considered fixed.
It also occurs on Freestyle project.
As per the attached screen shot, "Change 12264" submit is not run after "Change 12263" submit has been run. "Change 12264" submit will be run after "Change 12265" is submitted.
$ p4 edit a.txt
//test/a.txt#23 - opened for edit
$ p4 submit -d "test"
Submitting change 12263.
Locking 1 files ...
edit //test/a.txt#24
Change 12263 submitted.
$ p4 edit a.txt
//test/a.txt#24 - opened for edit
$ p4 submit -d "test"
Submitting change 12264.
Locking 1 files ...
edit //test/a.txt#25
Change 12264 submitted.
$ p4 edit a.txt
//test/a.txt#25 - opened for edit
$ p4 submit -d "test"
Submitting change 12265.
Locking 1 files ...
edit //test/a.txt#26
Change 12265 submitted.
When not using "Poll per change", concurrent submitted changelists are run together in one Jenkins build.
Hence, it would be good if one Jenkins build is run automatically per submit without remaining submitted changelists like "Poll per change".
Hope it makes sense.