Details
-
Type:
Bug
-
Status: Closed (View Workflow)
-
Priority:
Critical
-
Resolution: Duplicate
-
Component/s: blueocean-plugin
-
Labels:
-
Environment:64-bit Linux CentOS 7, Jenkins 2.77, plenty of plugins (including pipeline)
-
Similar Issues:
Description
We've been having scheduling problems on our Jenkins server in the past few weeks. I haven't been able to figure out what is causing these problems, but I've noticed the following things:
- Pipeline jobs are displayed as 'pending: waiting for executors on master', even though only 1 job is running, and there are 19 available executors.
- Triggering a (non-pipeline) job manually appears to have no effect (no job appears in the queue), until several minutes later.
- On both slaves and master, jobs are displayed as running, even though they have already finished minutes ago.(which can be seen by clicking on the job, where you can see that the job has been successful/failed).
I have not been able to pinpoint a reason for this. It doesn't appear to be related to load, as I've seen this issue with a single running (light-weight) job and 1 job in the queue, with low load and low disk access on the master.
This is making Jenkins (at the times when it occurs) unusable for us.
Attachments
Issue Links
- duplicates
-
-
- Closed
-
Activity
It seems the part of the log is missing (one below "Caused by: java.lang.NullPointerException"). Could you please provide it?
I think it's this (i'm getting the same)
Caused by: java.lang.NullPointerException
at io.jenkins.blueocean.service.embedded.rest.QueueUtil.getQueuedItem(QueueUtil.java:24)
at io.jenkins.blueocean.events.BlueMessageEnricher.enrich(BlueMessageEnricher.java:100)
at org.jenkinsci.plugins.pubsub.PubsubBus.publish(PubsubBus.java:133)
... 16 more
The stacktrace indicates that this problem is a duplicate of JENKINS-45589 and has been fixed.
Not sure if it's related, but the following stack trace occurs sometimes in our logs:
WARNING: Error publishing Run queued event.
org.jenkinsci.plugins.pubsub.MessageException: Event enrichment failure due to unexpected exception in io.jenkins.blueocean.events.BlueMessageEnricher.
{{ at org.jenkinsci.plugins.pubsub.PubsubBus.publish(PubsubBus.java:135)}}
{{ at org.jenkinsci.plugins.pubsub.listeners.SyncQueueListener.publish(SyncQueueListener.java:178)}}
{{ at org.jenkinsci.plugins.pubsub.listeners.SyncQueueListener.publish(SyncQueueListener.java:172)}}
{{ at org.jenkinsci.plugins.pubsub.listeners.SyncQueueListener.onEnterWaiting(SyncQueueListener.java:139)}}
{{ at hudson.model.Queue$WaitingItem.enter(Queue.java:2393)}}
{{ at hudson.model.Queue.scheduleInternal(Queue.java:632)}}
{{ at hudson.model.Queue.schedule2(Queue.java:588)}}
{{ at jenkins.model.ParameterizedJobMixIn.scheduleBuild2(ParameterizedJobMixIn.java:155)}}
{{ at jenkins.model.ParameterizedJobMixIn.scheduleBuild(ParameterizedJobMixIn.java:115)}}
{{ at jenkins.model.ParameterizedJobMixIn$ParameterizedJob.scheduleBuild(ParameterizedJobMixIn.java:405)}}
{{ at hudson.triggers.TimerTrigger.run(TimerTrigger.java:67)}}
{{ at hudson.triggers.Trigger.checkTriggers(Trigger.java:278)}}
{{ at hudson.triggers.Trigger$Cron.doRun(Trigger.java:226)}}
{{ at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:51)}}
{{ at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58)}}
{{ at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)}}
{{ at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)}}
{{ at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)}}
{{ at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)}}
{{ at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)}}
{{ at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)}}
{{ at java.lang.Thread.run(Thread.java:745)}}
Caused by: java.lang.NullPointerException
Searching around turns up this report: https://issues.jenkins-ci.org/browse/JENKINS-44014
Priority sorter is not present on our system though.