Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-44747

Queue locked indefinitely in ThrottleQueueTaskDispatcher.categoriesForPipeline

XMLWordPrintable

      See this thread dump. This call can block (for example while a Pipeline build is getting ready to resume), whereas this caller ought to return quickly (milliseconds, if at all possible).

      Observed in a Jenkins server after restarting with some running builds. Since all Queue calls hanged, the system seemed to be frozen—no branch indexing, etc. Even taking a fresh support bundle hanged for the same reason (probably support-core ought to interrupt component threads which do not finish within some fixed timeout). Cannot cancel running builds for the same reason. The only way to recover was to disable the TCB plugin.

      Have also heard a field report of a server running TCB 2.0 exhibiting similar symptoms, but have not yet confirmed duplicate, pending existence of a thread dump.

          [JENKINS-44747] Queue locked indefinitely in ThrottleQueueTaskDispatcher.categoriesForPipeline

          Andrew Bayer added a comment -

          Is there any way to determine whether the call for the FlowNode could block ahead of time?

          Andrew Bayer added a comment - Is there any way to determine whether the call for the FlowNode could block ahead of time?

          Jesse Glick added a comment -

          Filed PR 50. I would consider it more of a hot-patch to avoid the deadlock than a real fix—presumably the throttling would be ineffective if the timeout were in fact hit, which could cause other problems, though surely less severe.

          Jesse Glick added a comment - Filed PR 50. I would consider it more of a hot-patch to avoid the deadlock than a real fix—presumably the throttling would be ineffective if the timeout were in fact hit, which could cause other problems, though surely less severe.

          SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Jesse Glick
          Path:
          pom.xml
          src/main/java/hudson/plugins/throttleconcurrents/ThrottleQueueTaskDispatcher.java
          http://jenkins-ci.org/commit/throttle-concurrent-builds-plugin/a4cb745f7f616d5ffe3a212bb45f5c8612388157
          Log:
          JENKINS-44747 Limit the amount of time categoriesForPipeline can run.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: pom.xml src/main/java/hudson/plugins/throttleconcurrents/ThrottleQueueTaskDispatcher.java http://jenkins-ci.org/commit/throttle-concurrent-builds-plugin/a4cb745f7f616d5ffe3a212bb45f5c8612388157 Log: JENKINS-44747 Limit the amount of time categoriesForPipeline can run.

          SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Andrew Bayer
          Path:
          README.md
          http://jenkins-ci.org/commit/throttle-concurrent-builds-plugin/c22a7e781ac0e393691f9b03b2221b209ab9abfe
          Log:
          Adding disclaimer for JENKINS-44747 change.

          Compare: https://github.com/jenkinsci/throttle-concurrent-builds-plugin/compare/a4cb745f7f61^...c22a7e781ac0

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Andrew Bayer Path: README.md http://jenkins-ci.org/commit/throttle-concurrent-builds-plugin/c22a7e781ac0e393691f9b03b2221b209ab9abfe Log: Adding disclaimer for JENKINS-44747 change. Compare: https://github.com/jenkinsci/throttle-concurrent-builds-plugin/compare/a4cb745f7f61 ^...c22a7e781ac0

          SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Andrew Bayer
          Path:
          pom.xml
          src/main/java/hudson/plugins/throttleconcurrents/ThrottleJobProperty.java
          src/main/java/hudson/plugins/throttleconcurrents/ThrottleQueueTaskDispatcher.java
          http://jenkins-ci.org/commit/throttle-concurrent-builds-plugin/d105bb791364f3d8bfe42dcad448b14f9b0ed8b7
          Log:
          Merge pull request #50 from jglick/Queue-JENKINS-44747

          JENKINS-44747 Limit the amount of time categoriesForPipeline can run

          Compare: https://github.com/jenkinsci/throttle-concurrent-builds-plugin/compare/7c61f6119c67...d105bb791364

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Andrew Bayer Path: pom.xml src/main/java/hudson/plugins/throttleconcurrents/ThrottleJobProperty.java src/main/java/hudson/plugins/throttleconcurrents/ThrottleQueueTaskDispatcher.java http://jenkins-ci.org/commit/throttle-concurrent-builds-plugin/d105bb791364f3d8bfe42dcad448b14f9b0ed8b7 Log: Merge pull request #50 from jglick/Queue- JENKINS-44747 JENKINS-44747 Limit the amount of time categoriesForPipeline can run Compare: https://github.com/jenkinsci/throttle-concurrent-builds-plugin/compare/7c61f6119c67...d105bb791364

          SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Andrew Bayer
          Path:
          README.md
          http://jenkins-ci.org/commit/throttle-concurrent-builds-plugin/fc9f6db53d21ebf50447e20a6304323ce62f6a46
          Log:
          Merge pull request #51 from jenkinsci/jenkins-44747-disclaimer

          Disclaimer for JENKINS-44747 change.

          Compare: https://github.com/jenkinsci/throttle-concurrent-builds-plugin/compare/d105bb791364...fc9f6db53d21

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Andrew Bayer Path: README.md http://jenkins-ci.org/commit/throttle-concurrent-builds-plugin/fc9f6db53d21ebf50447e20a6304323ce62f6a46 Log: Merge pull request #51 from jenkinsci/jenkins-44747-disclaimer Disclaimer for JENKINS-44747 change. Compare: https://github.com/jenkinsci/throttle-concurrent-builds-plugin/compare/d105bb791364...fc9f6db53d21

          Andrew Bayer added a comment -

          Releasing as 2.0.1 as we speak.

          Andrew Bayer added a comment - Releasing as 2.0.1 as we speak.

          SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Oleg Nenashev
          Path:
          CHANGELOG.md
          http://jenkins-ci.org/commit/throttle-concurrent-builds-plugin/080f4768bfd31d55c717f904c223bfb6119a89fe
          Log:
          [JENKINS-44756, JENKINS-44747] - 2.0.1 changelog + Errata notice

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oleg Nenashev Path: CHANGELOG.md http://jenkins-ci.org/commit/throttle-concurrent-builds-plugin/080f4768bfd31d55c717f904c223bfb6119a89fe Log: [JENKINS-44756, JENKINS-44747] - 2.0.1 changelog + Errata notice

          SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Oleg Nenashev
          Path:
          CHANGELOG.md
          http://jenkins-ci.org/commit/throttle-concurrent-builds-plugin/d58cb2873a58ed59323b033aecd3c2536c569ad3
          Log:
          Merge pull request #52 from jenkinsci/changelog/2.0.1

          [JENKINS-44756, JENKINS-44747] - 2.0.1 changelog + Errata notice

          Compare: https://github.com/jenkinsci/throttle-concurrent-builds-plugin/compare/cfc775b523fe...d58cb2873a58

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oleg Nenashev Path: CHANGELOG.md http://jenkins-ci.org/commit/throttle-concurrent-builds-plugin/d58cb2873a58ed59323b033aecd3c2536c569ad3 Log: Merge pull request #52 from jenkinsci/changelog/2.0.1 [JENKINS-44756, JENKINS-44747] - 2.0.1 changelog + Errata notice Compare: https://github.com/jenkinsci/throttle-concurrent-builds-plugin/compare/cfc775b523fe...d58cb2873a58

            jglick Jesse Glick
            jglick Jesse Glick
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: