Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-44014

job triggering and queue broken since update to 2.57

    XMLWordPrintable

Details

    Description

      Since update to jenkins 2.57 (and still after update to 2.58), there is a lot of delay when clicking on 'build' button for a job. It could reach 5 minutes before really starting the build.

      Queue handling seems also to be altered (jobs don't come in queue rapidely)

      And the job triggering by upstream job is also really slow, reaching sometimes internal 'timeout'.

      What i can get from logs is (not sure which of those could be really related but it seems that those errors are linked together) :

      • NPE SyncQueueListener

      mai 03, 2017 12:42:02 PM org.jenkinsci.plugins.pubsub.listeners.SyncQueueListener publish
      WARNING: Error publishing Run queued event.
      org.jenkinsci.plugins.pubsub.MessageException: Event enrichment failure due to unexpected exception in io.jenkins.blueocean.events.BlueMessageEnricher.
      at org.jenkinsci.plugins.pubsub.PubsubBus.publish(PubsubBus.java:113)
      at org.jenkinsci.plugins.pubsub.listeners.SyncQueueListener.publish(SyncQueueListener.java:178)
      at org.jenkinsci.plugins.pubsub.listeners.SyncQueueListener.publish(SyncQueueListener.java:172)
      at org.jenkinsci.plugins.pubsub.listeners.SyncQueueListener.onEnterWaiting(SyncQueueListener.java:139)
      at hudson.model.Queue$WaitingItem.enter(Queue.java:2392)
      at hudson.model.Queue.scheduleInternal(Queue.java:633)
      at hudson.model.Queue.schedule2(Queue.java:589)
      at jenkins.model.ParameterizedJobMixIn.scheduleBuild2(ParameterizedJobMixIn.java:138)
      at jenkins.model.ParameterizedJobMixIn.scheduleBuild(ParameterizedJobMixIn.java:94)
      at hudson.model.AbstractProject.scheduleBuild(AbstractProject.java:838)
      at hudson.triggers.TimerTrigger.run(TimerTrigger.java:67)
      at hudson.triggers.Trigger.checkTriggers(Trigger.java:278)
      at hudson.triggers.Trigger$Cron.doRun(Trigger.java:226)
      at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:51)
      at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58)
      at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
      at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
      at java.lang.Thread.run(Unknown Source)
      Caused by: java.lang.NullPointerException

      • Warnings on AdvancedQueueSorterQueueListener

      mai 03, 2017 8:18:37 AM jenkins.advancedqueue.sorter.AdvancedQueueSorterQueueListener onEnterBuildable
      WARNING: onEnterBuilding() called without prior call to onEnterWaiting() for 'API-availability-dev'

      • Warnings on hudson.model.Queue$WaitingItem enter

      mai 03, 2017 8:25:11 AM hudson.model.Queue$WaitingItem enter
      WARNING: QueueListener failed while processing hudson.model.Queue$WaitingItem:ExecutorStepExecution.PlaceholderTask

      {runId=Certifications/Sparktacus/Development_Sparktacus#12,label=Certif-BDXBLD006,context=CpsStepContext[3:null]:OwnerCertifications/Sparktacus/Development_Sparktacus/12:Certifications/Sparktacus/Development_Sparktacus #12,cookie=8d0b6bd9-e673-409f-9234-2279205b290b}

      :69158
      java.lang.ClassCastException: org.jenkinsci.plugins.workflow.support.steps.ExecutorStepExecution$PlaceholderTask cannot be cast to hudson.model.Job
      at jenkins.advancedqueue.PriorityConfigurationPlaceholderTaskHelper.getPriority(PriorityConfigurationPlaceholderTaskHelper.java:19)
      at jenkins.advancedqueue.PriorityConfiguration.getPriorityInternal(PriorityConfiguration.java:213)
      at jenkins.advancedqueue.PriorityConfiguration.getPriority(PriorityConfiguration.java:205)
      at jenkins.advancedqueue.sorter.AdvancedQueueSorter.onNewItem(AdvancedQueueSorter.java:136)
      at jenkins.advancedqueue.sorter.AdvancedQueueSorterQueueListener.onEnterWaiting(AdvancedQueueSorterQueueListener.java:46)
      at hudson.model.Queue$WaitingItem.enter(Queue.java:2392)
      at hudson.model.Queue.scheduleInternal(Queue.java:633)
      at hudson.model.Queue.schedule2(Queue.java:589)
      at hudson.model.Queue.schedule2(Queue.java:709)
      at org.jenkinsci.plugins.workflow.support.pickles.ExecutorPickle$1.tryResolve(ExecutorPickle.java:79)
      at org.jenkinsci.plugins.workflow.support.pickles.ExecutorPickle$1.tryResolve(ExecutorPickle.java:73)
      at org.jenkinsci.plugins.workflow.support.pickles.TryRepeatedly$1.run(TryRepeatedly.java:92)
      at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58)
      at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
      at java.util.concurrent.FutureTask.run(Unknown Source)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown Source)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
      at java.lang.Thread.run(Unknown Source)

      • NPE on hudson.model.Queue$LeftItem enter

      mai 03, 2017 8:36:03 AM hudson.model.Queue$LeftItem enter
      WARNING: QueueListener failed while processing hudson.model.Queue$LeftItem:ExecutorStepExecution.PlaceholderTask

      {runId=Certifications/Sparktacus/Development_Sparktacus#12,label=Certif-BDXBLD006,context=CpsStepContext[3:null]:OwnerCertifications/Sparktacus/Development_Sparktacus/12:Certifications/Sparktacus/Development_Sparktacus #12,cookie=8d0b6bd9-e673-409f-9234-2279205b290b}

      :69158
      java.lang.NullPointerException
      at jenkins.advancedqueue.sorter.AdvancedQueueSorter.onLeft(AdvancedQueueSorter.java:145)
      at jenkins.advancedqueue.sorter.AdvancedQueueSorterQueueListener.onLeft(AdvancedQueueSorterQueueListener.java:51)
      at hudson.model.Queue$LeftItem.enter(Queue.java:2660)
      at hudson.model.Queue.onStartExecuting(Queue.java:1156)
      at hudson.model.Executor$1.call(Executor.java:352)
      at hudson.model.Executor$1.call(Executor.java:338)
      at hudson.model.Queue._withLock(Queue.java:1401)
      at hudson.model.Queue.withLock(Queue.java:1262)
      at hudson.model.Executor.run(Executor.java:338)

      Attachments

        Issue Links

          Activity

            splashnenen Alexandre Aubert created issue -
            splashnenen Alexandre Aubert made changes -
            Field Original Value New Value
            Priority Critical [ 2 ] Blocker [ 1 ]
            jglick Jesse Glick made changes -
            Component/s prioritysorter-plugin [ 15771 ]
            Component/s core [ 15593 ]
            Assignee Magnus Sandberg [ emsa23 ]
            splashnenen Alexandre Aubert made changes -
            Assignee Magnus Sandberg [ emsa23 ] Alexandre Aubert [ splashnenen ]
            splashnenen Alexandre Aubert made changes -
            Assignee Alexandre Aubert [ splashnenen ] Magnus Sandberg [ emsa23 ]
            splashnenen Alexandre Aubert made changes -
            Component/s workflow-durable-task-step-plugin [ 21715 ]
            Component/s prioritysorter-plugin [ 15771 ]
            Assignee Magnus Sandberg [ emsa23 ]
            jglick Jesse Glick made changes -
            Component/s prioritysorter-plugin [ 15771 ]
            Component/s workflow-durable-task-step-plugin [ 21715 ]
            Assignee Magnus Sandberg [ emsa23 ]
            oleg_nenashev Oleg Nenashev made changes -
            Assignee Magnus Sandberg [ emsa23 ] Oleg Nenashev [ oleg_nenashev ]
            oleg_nenashev Oleg Nenashev made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            oleg_nenashev Oleg Nenashev made changes -
            Status In Progress [ 3 ] In Review [ 10005 ]
            oleg_nenashev Oleg Nenashev made changes -
            Remote Link This issue links to "PR #39 (Web Link)" [ 17147 ]
            oleg_nenashev Oleg Nenashev made changes -
            Resolution Fixed [ 1 ]
            Status In Review [ 10005 ] Resolved [ 5 ]
            markewaite Mark Waite made changes -
            Status Resolved [ 5 ] Closed [ 6 ]

            People

              oleg_nenashev Oleg Nenashev
              splashnenen Alexandre Aubert
              Votes:
              2 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: