Threads blocked in /blue/rest/organizations/jenkins/computers/ due to loop in PipelineRunImpl.getCommitUrl

This issue is archived. You can view it, but you can't modify it. Learn more

XMLWordPrintable

      ci.jenkins.io froze, and danielbeck found that hundreds of threads were blocked in

      "Handling GET /blue/rest/organizations/jenkins/computers/ from ... : ..." #... prio=5 os_prio=0 cpu=9992.49ms elapsed=2517.82s tid=... nid=... waiting for monitor entry  [...]
         java.lang.Thread.State: BLOCKED (on object monitor)
      	at jenkins.model.lazy.AbstractLazyLoadRunMap.getByNumber(AbstractLazyLoadRunMap.java:370)
      	- waiting to lock <...> (a hudson.model.RunMap)
      	at jenkins.model.lazy.LazyBuildMixIn.getBuildByNumber(LazyBuildMixIn.java:228)
      	at org.jenkinsci.plugins.workflow.job.WorkflowJob.getBuildByNumber(WorkflowJob.java:232)
      	at io.jenkins.blueocean.rest.impl.pipeline.PipelineRunImpl.getCommitUrl(PipelineRunImpl.java:197)
      	at java.lang.invoke.DirectMethodHandle$Holder.invokeVirtual(java.base@11.0.4/DirectMethodHandle$Holder)
      	at java.lang.invoke.LambdaForm$MH/....invoke(java.base@11.0.4/LambdaForm$MH)
      	at java.lang.invoke.LambdaForm$MH/....invoke_MT(java.base@11.0.4/LambdaForm$MH)
      	at io.jenkins.blueocean.commons.stapler.export.MethodProperty.getValue(MethodProperty.java:72)
      	at io.jenkins.blueocean.commons.stapler.export.ExportInterceptor$1.getValue(ExportInterceptor.java:46)
      	at io.jenkins.blueocean.commons.stapler.Export$BlueOceanExportInterceptor.getValue(Export.java:196)
      	at io.jenkins.blueocean.commons.stapler.export.Property.writeTo(Property.java:136)
      	at io.jenkins.blueocean.commons.stapler.export.Model.writeNestedObjectTo(Model.java:228)
      	at io.jenkins.blueocean.commons.stapler.export.Property.writeValue(Property.java:303)
      	at io.jenkins.blueocean.commons.stapler.export.Property.writeValue(Property.java:169)
      	at io.jenkins.blueocean.commons.stapler.export.Property.writeTo(Property.java:154)
      	at io.jenkins.blueocean.commons.stapler.export.Model.writeNestedObjectTo(Model.java:228)
      	at io.jenkins.blueocean.commons.stapler.export.Property.writeValue(Property.java:303)
      	at io.jenkins.blueocean.commons.stapler.export.Property.writeBuffered(Property.java:175)
      	at io.jenkins.blueocean.commons.stapler.export.Property.writeValue(Property.java:235)
      	at io.jenkins.blueocean.commons.stapler.export.Property.writeValue(Property.java:169)
      	at io.jenkins.blueocean.commons.stapler.export.Property.writeTo(Property.java:154)
      	at io.jenkins.blueocean.commons.stapler.export.Model.writeNestedObjectTo(Model.java:228)
      	at io.jenkins.blueocean.commons.stapler.export.Property.writeValue(Property.java:303)
      	at io.jenkins.blueocean.commons.stapler.export.Property.writeBuffered(Property.java:175)
      	at io.jenkins.blueocean.commons.stapler.export.Property.writeValue(Property.java:219)
      	at io.jenkins.blueocean.commons.stapler.export.Property.writeValue(Property.java:169)
      	at io.jenkins.blueocean.commons.stapler.export.Property.writeTo(Property.java:154)
      	at io.jenkins.blueocean.commons.stapler.export.Model.writeNestedObjectTo(Model.java:228)
      	at io.jenkins.blueocean.commons.stapler.export.Model.writeTo(Model.java:199)
      	at io.jenkins.blueocean.commons.stapler.Export.writeOne(Export.java:177)
      	at io.jenkins.blueocean.commons.stapler.Export.serveExposedBean(Export.java:168)
      	at io.jenkins.blueocean.commons.stapler.Export.doJson(Export.java:108)
      	at io.jenkins.blueocean.commons.stapler.TreeResponse$Processor$1.generateResponse(TreeResponse.java:48)
      	at ...
      

      Looking at the code, this seems like a classic lazy-loading violation: looping over possibly every historical build of a job, each load of which involves expensive disk I/O. Under no circumstances may a plugin do this.

      This seems to have been noted in JENKINS-56773 but the response was just to disable the blueocean-executor-info plugin. How are users who already have this plugin installed supposed to know this? Some throttling was added, but evidently not enough.

            Assignee:
            Jesse Glick
            Reporter:
            Jesse Glick
            Archiver:
            Jenkins Service Account

              Created:
              Updated:
              Archived: