-
Bug
-
Resolution: Fixed
-
Blocker
-
None
-
Priority Sorter 3.0, CloudBees Folder 4.7
Jenkins 1.612
running under Tomcat 8 on JDK 1.8.45 64Bit, Windows Server 2008
After upgrade to Priority Sorter 3.0 and defining the new Priority Groups Jenkins doesn't schedule any more jobs but just throws NPE:
May 04, 2015 8:08:10 AM SEVERE hudson.triggers.SafeTimerTask run Timer task hudson.model.Queue$MaintainTask@eccba45 failed java.lang.NullPointerException at jenkins.advancedqueue.sorter.AdvancedQueueSorter$2.compare(AdvancedQueueSorter.java:80) at jenkins.advancedqueue.sorter.AdvancedQueueSorter$2.compare(AdvancedQueueSorter.java:76) at java.util.TimSort.countRunAndMakeAscending(TimSort.java:351) at java.util.TimSort.sort(TimSort.java:216) at java.util.Arrays.sort(Arrays.java:1512) at java.util.ArrayList.sort(ArrayList.java:1454) at java.util.Collections.sort(Collections.java:175) at jenkins.advancedqueue.sorter.AdvancedQueueSorter.sortBuildableItems(AdvancedQueueSorter.java:76) at hudson.model.Queue.maintain(Queue.java:1342) at hudson.model.Queue$MaintainTask.doRun(Queue.java:2457) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:51) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
Priority Sorter Log itself looks very much normal and priorities seem to be assigned as expected. There are no exceptions in that log:
[...]
New Item: Id: 20542, JobName: XXX-Sync-Rsync-Repository with ..., jobGroupId: 2, reason: <none>, priority: 1, weight: 1.0, status: WAITING
Evaluating JobGroup [0] ...
Evaluating JobGroup [1] ...
Evaluating JobGroup [2] ...
Evaluating strategies ...
Evaluating strategy [Job Triggered by a Upstream Build] ...
Evaluating strategy [Take the priority from Property on the Job] ...
No applicable strategy - Using JobGroup default
May 04, 2015 8:07:02 AM FINE jenkins.advancedqueue.ItemTransitionLogger logBuilableItem
Buildable: Id: 20542, JobName: ***-Sync-Rsync-Repository with ***, jobGroupId: 2, reason: <none>, priority: 1, weight: 1.0, status: BUILDABLE
May 04, 2015 8:11:02 AM FINER PrioritySorter.Queue.Items
New Item: Id: 20543, JobName: ADMIN_***_Install_RTS_***-slave8, jobGroupId: 6, reason: <none>, priority: 3, weight: 3.0, status: WAITING
Evaluating JobGroup [0] ...
Evaluating JobGroup [1] ...
Evaluating JobGroup [2] ...
Evaluating JobGroup [3] ...
Evaluating JobGroup [4] ...
Evaluating JobGroup [5] ...
Checking for Job Property inclusion for [*** Unit Tests]...
Checking for Cloudbees Folder inclusion ...
Evaluating Folder [ADMIN_***_Install_RTS] ...
No match ...
Evaluating JobGroup [6] ...
Evaluating strategies ...
Evaluating strategy [Job Triggered by a Upstream Build] ...
Evaluating strategy [Take the priority from Property on the Job] ...
No applicable strategy - Using JobGroup default
[...]
Downgrade to Priority Sorter 2.12 removes the problem.
[JENKINS-28195] Jenkins not scheduling any jobs after upgrade to Priority Sorter 3.0 but giving exception
Kurt
created issue -
Kurt
made changes -
| Description |
Original:
After upgrade to Priority Sorter 3.0 and defining the new Priority Groups Jenkins doesn't schedule any more jobs but just throws NPE: {noformat} May 04, 2015 8:08:10 AM SEVERE hudson.triggers.SafeTimerTask run Timer task hudson.model.Queue$MaintainTask@eccba45 failed java.lang.NullPointerException at jenkins.advancedqueue.sorter.AdvancedQueueSorter$2.compare(AdvancedQueueSorter.java:80) at jenkins.advancedqueue.sorter.AdvancedQueueSorter$2.compare(AdvancedQueueSorter.java:76) at java.util.TimSort.countRunAndMakeAscending(TimSort.java:351) at java.util.TimSort.sort(TimSort.java:216) at java.util.Arrays.sort(Arrays.java:1512) at java.util.ArrayList.sort(ArrayList.java:1454) at java.util.Collections.sort(Collections.java:175) at jenkins.advancedqueue.sorter.AdvancedQueueSorter.sortBuildableItems(AdvancedQueueSorter.java:76) at hudson.model.Queue.maintain(Queue.java:1342) at hudson.model.Queue$MaintainTask.doRun(Queue.java:2457) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:51) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) {noformat} Priority Sorter Log itself looks very much normal and priorities seem to be assigned as expected. There are no exceptions in that log: {noformat} [...] New Item: Id: 20542, JobName: XXX-Sync-Rsync-Repository with ..., jobGroupId: 2, reason: <none>, priority: 1, weight: 1.0, status: WAITING Evaluating JobGroup [0] ... Evaluating JobGroup [1] ... Evaluating JobGroup [2] ... Evaluating strategies ... Evaluating strategy [Job Triggered by a Upstream Build] ... Evaluating strategy [Take the priority from Property on the Job] ... No applicable strategy - Using JobGroup default May 04, 2015 8:07:02 AM FINE jenkins.advancedqueue.ItemTransitionLogger logBuilableItem Buildable: Id: 20542, JobName: ***-Sync-Rsync-Repository with ***, jobGroupId: 2, reason: <none>, priority: 1, weight: 1.0, status: BUILDABLE May 04, 2015 8:11:02 AM FINER PrioritySorter.Queue.Items New Item: Id: 20543, JobName: ADMIN_***_Install_RTS_***-slave8, jobGroupId: 6, reason: <none>, priority: 3, weight: 3.0, status: WAITING Evaluating JobGroup [0] ... Evaluating JobGroup [1] ... Evaluating JobGroup [2] ... Evaluating JobGroup [3] ... Evaluating JobGroup [4] ... Evaluating JobGroup [5] ... Checking for Job Property inclusion for [*** Unit Tests]... Checking for Cloudbees Folder inclusion ... Evaluating Folder [ADMIN_***_Install_RTS] ... No match ... Evaluating JobGroup [6] ... Evaluating strategies ... Evaluating strategy [Job Triggered by a Upstream Build] ... Evaluating strategy [Take the priority from Property on the Job] ... No applicable strategy - Using JobGroup default [...] {noformat} |
New:
After upgrade to Priority Sorter 3.0 and defining the new Priority Groups Jenkins doesn't schedule any more jobs but just throws NPE: {noformat} May 04, 2015 8:08:10 AM SEVERE hudson.triggers.SafeTimerTask run Timer task hudson.model.Queue$MaintainTask@eccba45 failed java.lang.NullPointerException at jenkins.advancedqueue.sorter.AdvancedQueueSorter$2.compare(AdvancedQueueSorter.java:80) at jenkins.advancedqueue.sorter.AdvancedQueueSorter$2.compare(AdvancedQueueSorter.java:76) at java.util.TimSort.countRunAndMakeAscending(TimSort.java:351) at java.util.TimSort.sort(TimSort.java:216) at java.util.Arrays.sort(Arrays.java:1512) at java.util.ArrayList.sort(ArrayList.java:1454) at java.util.Collections.sort(Collections.java:175) at jenkins.advancedqueue.sorter.AdvancedQueueSorter.sortBuildableItems(AdvancedQueueSorter.java:76) at hudson.model.Queue.maintain(Queue.java:1342) at hudson.model.Queue$MaintainTask.doRun(Queue.java:2457) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:51) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) {noformat} Priority Sorter Log itself looks very much normal and priorities seem to be assigned as expected. There are no exceptions in that log: {noformat} [...] New Item: Id: 20542, JobName: XXX-Sync-Rsync-Repository with ..., jobGroupId: 2, reason: <none>, priority: 1, weight: 1.0, status: WAITING Evaluating JobGroup [0] ... Evaluating JobGroup [1] ... Evaluating JobGroup [2] ... Evaluating strategies ... Evaluating strategy [Job Triggered by a Upstream Build] ... Evaluating strategy [Take the priority from Property on the Job] ... No applicable strategy - Using JobGroup default May 04, 2015 8:07:02 AM FINE jenkins.advancedqueue.ItemTransitionLogger logBuilableItem Buildable: Id: 20542, JobName: ***-Sync-Rsync-Repository with ***, jobGroupId: 2, reason: <none>, priority: 1, weight: 1.0, status: BUILDABLE May 04, 2015 8:11:02 AM FINER PrioritySorter.Queue.Items New Item: Id: 20543, JobName: ADMIN_***_Install_RTS_***-slave8, jobGroupId: 6, reason: <none>, priority: 3, weight: 3.0, status: WAITING Evaluating JobGroup [0] ... Evaluating JobGroup [1] ... Evaluating JobGroup [2] ... Evaluating JobGroup [3] ... Evaluating JobGroup [4] ... Evaluating JobGroup [5] ... Checking for Job Property inclusion for [*** Unit Tests]... Checking for Cloudbees Folder inclusion ... Evaluating Folder [ADMIN_***_Install_RTS] ... No match ... Evaluating JobGroup [6] ... Evaluating strategies ... Evaluating strategy [Job Triggered by a Upstream Build] ... Evaluating strategy [Take the priority from Property on the Job] ... No applicable strategy - Using JobGroup default [...] {noformat} Downgrade to Priority Sorter 2.12 removes the problem. |
Plugins can only be updated after a restart.
However when shutting down Jenkins cleanly, it saves the queue, and loads it upon restart.
Kurt
added a comment - Yes, I had restart my Jenkins for updating the plugin.
But yes: In my case there had been saved items in the queue from the time before shut down. Just like commented by Daniel.
Kurt
added a comment - Yes, I had restart my Jenkins for updating the plugin.
But yes: In my case there had been saved items in the queue from the time before shut down. Just like commented by Daniel. Yes, the questions was rather if you had items in the queue in this specific situation. Just trying to figure out where the glitch is.
I obviously need to handle this more gracefully but I would also like to understand where it fails because it shouldn't be possible to happen.
My guess is that this is a start-up issue and if you empty the queue and re-queue your jobs after restart it will work. Just a guess, educated, but still ...
Kurt
added a comment - Thanks Magnus. I will give purging the queue and then updating a try tomorrow morning in our Jenkins maintenance window (7-8 CEST).
Please also let me know if you need any other information and I'll try to provide it.
Kurt
added a comment - Thanks Magnus. I will give purging the queue and then updating a try tomorrow morning in our Jenkins maintenance window (7-8 CEST).
Please also let me know if you need any other information and I'll try to provide it. It would be great if you have the possibility to test with an empty queue, hopefully it works.
The problem might (probably will) reoccur on each restart with existing queue so I will fix this as soon as I can.
Thanks.
Kurt
added a comment - I can confirm that when restarting Jenkins with an empty build queue the problem does not occur and Priority Sorter v 3.0 works as expected (as of now for at least 1 hour).
I'll stay with 3.0 version for now and try to remember to purge the queue before shutting down 
Thanks for your quick help. It would be great if a fix could be included in next version of Priority Sorter.
Kurt
added a comment - I can confirm that when restarting Jenkins with an empty build queue the problem does not occur and Priority Sorter v 3.0 works as expected (as of now for at least 1 hour).
I'll stay with 3.0 version for now and try to remember to purge the queue before shutting down
Thanks for your quick help. It would be great if a fix could be included in next version of Priority Sorter. 
Matthew Webber
added a comment - I upgraded to 3.0 with an empty queue, and I started seeing the problem as soon as I started manually triggering jobs (as I recall, the first job started ok, but subsequent ones did not). Reverting fixed the problem.
Unfortunately I'm off on holidays at the moment, so I can't retest, but that might be an additional data item for you.
Matthew Webber
added a comment - I upgraded to 3.0 with an empty queue, and I started seeing the problem as soon as I started manually triggering jobs (as I recall, the first job started ok, but subsequent ones did not). Reverting fixed the problem.
Unfortunately I'm off on holidays at the moment, so I can't retest, but that might be an additional data item for you.
What happens is that the sorter is instructed to sort items it has not seen before, the sorter should be informed of the items existence as it enters the queue.
Did you start with an empty queue after the update or did you have items in the queue when you updated?
Did you restart Jenkins after the update?