-
Bug
-
Resolution: Fixed
-
Major
-
None
-
Powered by SuggestiMate
Job A triggers job B and passes its git hash along. Job A is fast but job B is slow. So there are times when job A finishes and triggers job B while job B is still running and processing the earlier git hash. When job B finishes its current run, the next run would be triggered by multiple runs of job A. The parameterized trigger passes the git hash from the earliest trigger by job A. As a result, git hashes from the later triggers are never processed by job B. The parameterized trigger should pass the git hash from the latest trigger by job A.
[JENKINS-15160] earliest git hash passed by parameterized build trigger when downstream job was triggered multiple times
I have experienced this issue in various versions of jenkins and parameterized trigger plugin. I have been upgrading both in the far chance that it might have been fixed from work on other bugs for the plugin. I am now using the latest versions: 1.486 for jenkins and 3.16 for parameterized trigger plugin and I'm still seeing the problem.
Indeed Job A triggers job B through the post build parameterized trigger plugin using Pass-through Git commit that was built. This is one example of the problem that I saw as recent as in the last hour:
Job B was triggered by both #414 and #415 of job A:
Started by upstream project A build number 414
originally caused by:
Started by remote host 127.0.0.1
Started by upstream project A build number 415
originally caused by:
Started by remote host 127.0.0.1
Revision: 495765555ce4db911825ea0b40bca6278669ae72
Note that the git commit hash of 4957 was actually from #414 of job A:
Build #414 (Oct 22, 2012 12:28:07 PM)
...
Started by remote host 127.0.0.1
Revision: 495765555ce4db911825ea0b40bca6278669ae72
The git commit hash of 4c41 from #415 was not what the triggered job B is building ... in fact, this commit never will get built if no more pushes come in:
Build #415 (Oct 22, 2012 1:04:17 PM)
...
Revision: 4c41dd60bdff7557133ebfe5229d7f9d06722cef
As a result, the developers (justifiably) complain that the latest build artifacts often do not reflect the latest push. Since the continuous integration system is broken, I have to continually monitor the jobs and manually kick them when necessary.
Ideally, there would be a checkbox for the user to indicate whether s/he wants every commit built, or just the latest commit built. In my case, job B runs many times slower than job A. If every commit is built, it would take days for job B to catch up. That would mean I would need to manually kill some runs in job B to help it catching up. constant manual intervention is very undesirable.
Pull request created to solve the issue
https://github.com/jenkinsci/git-plugin/pull/107
Includes the option to combine commits, if required.
The default is to create separate builds for each build triggered with a git hash.
Thanks. Will this also solve the problem that only the git commit from the earliest trigger in a multiply triggered downstream job gets processed? In my example above, commit 4c41 from #415 never gets built (until I manually kick the job).
Code changed in jenkins
User: cjo9900
Path:
src/main/java/hudson/plugins/git/RevisionParameterAction.java
src/test/java/hudson/plugins/git/RevisionParameterActionTest.java
http://jenkins-ci.org/commit/git-plugin/0d21dcc3cbfce9e14b117560e11703a267cc0f1f
Log:
[FIXED JENKINS-15160] Earliest git hash passed by parameter trigger.
Correction is to create a seperate build for each git hash rather than
allowing them to be combined into a single build.
Code changed in jenkins
User: cjo9900
Path:
src/main/java/hudson/plugins/git/GitRevisionBuildParameters.java
src/main/java/hudson/plugins/git/RevisionParameterAction.java
src/main/resources/hudson/plugins/git/GitRevisionBuildParameters/config.jelly
src/main/resources/hudson/plugins/git/GitRevisionBuildParameters/help-combineQueuedCommits.html
src/test/java/hudson/plugins/git/RevisionParameterActionTest.java
http://jenkins-ci.org/commit/git-plugin/e442a7904904256ff9aa79e885c085cacd514ede
Log:
JENKINS-15160 add support for combining git hashs when passed downstream
Don't combine with manually started builds
Code changed in jenkins
User: Nicolas De loof
Path:
src/main/java/hudson/plugins/git/GitRevisionBuildParameters.java
src/main/java/hudson/plugins/git/RevisionParameterAction.java
src/main/resources/hudson/plugins/git/GitRevisionBuildParameters/config.jelly
src/main/resources/hudson/plugins/git/GitRevisionBuildParameters/help-combineQueuedCommits.html
src/test/java/hudson/plugins/git/RevisionParameterActionTest.java
http://jenkins-ci.org/commit/git-plugin/17132ba7e8b2323cdb5b09ee363afbe9c3042b65
Log:
Merge pull request #107 from cjo9900/JENKINS-15160
Jenkins 15160
Compare: https://github.com/jenkinsci/git-plugin/compare/513d89b4a8a9...17132ba7e8b2
Many thanks. Fix Version/s for this bug says None. Which version contains the fix?
Should combineQueuedCommits=true result in a single long downstream build (even if its upstream ran multiple times)? If so, it does not appear to work for me.
My simple test with Jenkins 1.480.2 (LTE), Git plugin 1.1.26, Paramtrigger plugin: 2.16:
1) new job: try-combinecommits-downstream
scm: a tiny git repo
build step: shell "sleep 120"
2) new job: try-combinecommits-base,
scm: the same repo, trigger by push notification
build step: shell "sleep 10"
trigger parametrized build: try-combinecommits-downstream,
param: Git SHA1
combineQueuedCommits: yes
3) a simple bash loop to pushed dummy changes dozens of times
Result: the short upstream job triggers the downsteam one for each build, flooding the queue - no sign of "combining" them, ie. to pick the last build. I'd expect a second trigger should remove the first one off the queue.
Do I miss anything? Thanks.
combineQueuedCommits=true does not seem to work => reopening.
See my previous comment.
Apologies, looks like I missed passing that flag to the Action that is created, so it never took effect.
Code changed in jenkins
User: cjo9900
Path:
src/main/java/hudson/plugins/git/GitRevisionBuildParameters.java
http://jenkins-ci.org/commit/git-plugin/951517ccd2b99a8a4464b444e03976a1ff958239
Log:
[FIXED JENKINS-15160] Pass combineCommits to action
Code changed in jenkins
User: Nicolas De loof
Path:
src/main/java/hudson/plugins/git/GitRevisionBuildParameters.java
http://jenkins-ci.org/commit/git-plugin/1349eaba9c5767e9bf52ab27b6c31681f0c13733
Log:
Merge pull request #129 from cjo9900/JENKINS-15160-2
[FIXED JENKINS-15160] Pass combineCommits to action
Compare: https://github.com/jenkinsci/git-plugin/compare/c72b83216954...1349eaba9c57
–
You received this message because you are subscribed to the Google Groups "Jenkins Commits" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-commits+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Thank you very much for this - it would fix an important regression for me (my long running tests are flooding the queue with the current version - this started to happen with the latest upgrade).
Any way to release it soon?
Indeed I'm also still having this problem. Have to manually kill some of the jobs in the long queue. Quite a hassle. Please release the fix soon.
Which version of jenkins/ parameterized trigger are you using?
How is Job A configured?
Is the parameterized trigger done as a post build trigger?
Are the git hashes parameters passed using "Pass-Through Git commit that was built" Parameters?
If the answers to above are both yes, then the issue seems to be that the action RevisionParameterAction[1] that the git-plugin is providing via the GitRevisionBuildParameters[2] class does not support the QueueAction[3] interface that would allow the Queue class to add separate builds for the different revisions.
Alternatively the issue is that the RevisionParameterAction[1] does not implement the FoldableAction[4] interface, that would allow the multiple revisions to be merged together so only the latest revision is passed to the next build, as shown in the Queue Class[5].
In most use cases the behaviour would want to be trigger the downstream job for every commit Job A built.
In a smaller number of cases there might be the need for the downstream job to run only for the latest commit in Job A, (not sure how that would work if Job A worked on multiple branches).
[1] https://github.com/jenkinsci/git-plugin/blob/master/src/main/java/hudson/plugins/git/RevisionParameterAction.java
[2] https://github.com/jenkinsci/git-plugin/blob/master/src/main/java/hudson/plugins/git/GitRevisionBuildParameters.java
[3] https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/model/Queue.java#L1467
[4] https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/model/queue/FoldableAction.java
[5] https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/model/Queue.java#L526