-
Bug
-
Resolution: Fixed
-
Blocker
-
None
-
Priority Sorter 3.0, CloudBees Folder 4.7
Jenkins 1.612
running under Tomcat 8 on JDK 1.8.45 64Bit, Windows Server 2008
After upgrade to Priority Sorter 3.0 and defining the new Priority Groups Jenkins doesn't schedule any more jobs but just throws NPE:
May 04, 2015 8:08:10 AM SEVERE hudson.triggers.SafeTimerTask run Timer task hudson.model.Queue$MaintainTask@eccba45 failed java.lang.NullPointerException at jenkins.advancedqueue.sorter.AdvancedQueueSorter$2.compare(AdvancedQueueSorter.java:80) at jenkins.advancedqueue.sorter.AdvancedQueueSorter$2.compare(AdvancedQueueSorter.java:76) at java.util.TimSort.countRunAndMakeAscending(TimSort.java:351) at java.util.TimSort.sort(TimSort.java:216) at java.util.Arrays.sort(Arrays.java:1512) at java.util.ArrayList.sort(ArrayList.java:1454) at java.util.Collections.sort(Collections.java:175) at jenkins.advancedqueue.sorter.AdvancedQueueSorter.sortBuildableItems(AdvancedQueueSorter.java:76) at hudson.model.Queue.maintain(Queue.java:1342) at hudson.model.Queue$MaintainTask.doRun(Queue.java:2457) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:51) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
Priority Sorter Log itself looks very much normal and priorities seem to be assigned as expected. There are no exceptions in that log:
[...]
New Item: Id: 20542, JobName: XXX-Sync-Rsync-Repository with ..., jobGroupId: 2, reason: <none>, priority: 1, weight: 1.0, status: WAITING
Evaluating JobGroup [0] ...
Evaluating JobGroup [1] ...
Evaluating JobGroup [2] ...
Evaluating strategies ...
Evaluating strategy [Job Triggered by a Upstream Build] ...
Evaluating strategy [Take the priority from Property on the Job] ...
No applicable strategy - Using JobGroup default
May 04, 2015 8:07:02 AM FINE jenkins.advancedqueue.ItemTransitionLogger logBuilableItem
Buildable: Id: 20542, JobName: ***-Sync-Rsync-Repository with ***, jobGroupId: 2, reason: <none>, priority: 1, weight: 1.0, status: BUILDABLE
May 04, 2015 8:11:02 AM FINER PrioritySorter.Queue.Items
New Item: Id: 20543, JobName: ADMIN_***_Install_RTS_***-slave8, jobGroupId: 6, reason: <none>, priority: 3, weight: 3.0, status: WAITING
Evaluating JobGroup [0] ...
Evaluating JobGroup [1] ...
Evaluating JobGroup [2] ...
Evaluating JobGroup [3] ...
Evaluating JobGroup [4] ...
Evaluating JobGroup [5] ...
Checking for Job Property inclusion for [*** Unit Tests]...
Checking for Cloudbees Folder inclusion ...
Evaluating Folder [ADMIN_***_Install_RTS] ...
No match ...
Evaluating JobGroup [6] ...
Evaluating strategies ...
Evaluating strategy [Job Triggered by a Upstream Build] ...
Evaluating strategy [Take the priority from Property on the Job] ...
No applicable strategy - Using JobGroup default
[...]
Downgrade to Priority Sorter 2.12 removes the problem.
[JENKINS-28195] Jenkins not scheduling any jobs after upgrade to Priority Sorter 3.0 but giving exception
Matthew Webber
added a comment - I upgraded to 3.0 with an empty queue, and I started seeing the problem as soon as I started manually triggering jobs (as I recall, the first job started ok, but subsequent ones did not). Reverting fixed the problem.
Unfortunately I'm off on holidays at the moment, so I can't retest, but that might be an additional data item for you.
Matthew Webber
added a comment - I upgraded to 3.0 with an empty queue, and I started seeing the problem as soon as I started manually triggering jobs (as I recall, the first job started ok, but subsequent ones did not). Reverting fixed the problem.
Unfortunately I'm off on holidays at the moment, so I can't retest, but that might be an additional data item for you. I will definitely add a more graceful handling when this issue occurs.
Unfortunately I cannot get to this until end of the week
Kurt
added a comment - Up to now (during the whole morning) I was not able to reproduce the "manual job trigger" issue reported in comment-227565.
I've at least started 4 manual job executions and I'm sure some other colleagues had also triggered some more.
Our settings and environment:
- We've only triggered via Web UI.
- We're only using priority strategies "Job Triggered by a Upstream Build" and "Take the priority from Property on the Job" for additional rules.
- All Job Groups are selecting jobs based on CloudBees folders.
- For "backward-compatibility" I've put one final "Catch-all" group at the very end of the Job Group list which uses "All Jobs" for include rule with priority "-- use default priority --" and additional rule "Job Triggered by a Upstream Build" and "Take the priority from Property on the Job".
Kurt
added a comment - Up to now (during the whole morning) I was not able to reproduce the "manual job trigger" issue reported in comment-227565 .
I've at least started 4 manual job executions and I'm sure some other colleagues had also triggered some more.
Our settings and environment:
We've only triggered via Web UI.
We're only using priority strategies "Job Triggered by a Upstream Build" and "Take the priority from Property on the Job" for additional rules.
All Job Groups are selecting jobs based on CloudBees folders.
For "backward-compatibility" I've put one final "Catch-all" group at the very end of the Job Group list which uses "All Jobs" for include rule with priority "-- use default priority --" and additional rule "Job Triggered by a Upstream Build" and "Take the priority from Property on the Job".
Code changed in jenkins
User: emsa23
Path:
src/main/java/jenkins/advancedqueue/PrioritySorterPlugin.java
http://jenkins-ci.org/commit/priority-sorter-plugin/fdfe38d01a241a70b0bc2e45274adcad7bb150b7
Log:
Trying to make start-up more stable
Code changed in jenkins
User: emsa23
Path:
src/main/java/jenkins/advancedqueue/sorter/AdvancedQueueSorter.java
http://jenkins-ci.org/commit/priority-sorter-plugin/b6bf3e4325a801146b40d959045b0f6f2652a96c
Log:
Gracefully fallback to queue-time sorting if items are unknown
Code changed in jenkins
User: emsa23
Path:
src/main/java/jenkins/advancedqueue/sorter/AdvancedQueueSorterQueueListener.java
http://jenkins-ci.org/commit/priority-sorter-plugin/cbf68049874f3efc8e71361bb49cf34e8091de43
Log:
Gracefully handle out-of-sync lifecycle events
Compare: https://github.com/jenkinsci/priority-sorter-plugin/compare/cc14f2ee194b...cbf68049874f
Code changed in jenkins
User: Oliver Gondža
Path:
src/main/java/jenkins/advancedqueue/PrioritySorterPlugin.java
http://jenkins-ci.org/commit/priority-sorter-plugin/12f7073dafd94301b0fe4e7ec987adbda30cdab2
Log:
JENKINS-28195 fixup cbf6804
Code changed in jenkins
User: emsa23
Path:
src/main/java/jenkins/advancedqueue/PrioritySorterPlugin.java
http://jenkins-ci.org/commit/priority-sorter-plugin/c9cca9f92835e9ec52089285743f8077092323b0
Log:
Merge pull request #26 from olivergondza/fix-startup
JENKINS-28195 fixup cbf6804
Compare: https://github.com/jenkinsci/priority-sorter-plugin/compare/6592055dee93...c9cca9f92835
I can confirm that when restarting Jenkins with an empty build queue the problem does not occur and Priority Sorter v 3.0 works as expected (as of now for at least 1 hour).
I'll stay with 3.0 version for now and try to remember to purge the queue before shutting down
Thanks for your quick help. It would be great if a fix could be included in next version of Priority Sorter.