Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-26201

Unreleased workspace locks after SCM polling

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      A user reports many builds stuck waiting for a lock on the project's primary workspace (not @2, etc.), which should only be possible if some code has acquired a "quick" lease on the workspace but then failed to ever release it. There are no threads shown holding the workspace lock. Several threads in AbstractProject.pollWithWorkspace and elsewhere are waiting for it, in some cases for days.

      https://github.com/jenkinsci/jenkins/blob/jenkins-1.580.1/core/src/main/java/hudson/model/AbstractProject.java#L1442-1445 shows a potential window of vulnerability: if getBuiltOn, createLauncher, decorateByEnv, or getEnvironment threw an exception, and it went unreported or was somehow lost in the logs, the lease could be left locked indefinitely.

        Attachments

          Activity

          jglick Jesse Glick created issue -
          jglick Jesse Glick made changes -
          Field Original Value New Value
          Status Open [ 1 ] In Progress [ 3 ]
          scm_issue_link SCM/JIRA link daemon made changes -
          Resolution Fixed [ 1 ]
          Status In Progress [ 3 ] Resolved [ 5 ]
          jglick Jesse Glick made changes -
          Labels lock polling robustness workspace lock lts-candidate polling robustness workspace
          jglick Jesse Glick made changes -
          Description A user reports many builds stuck waiting for a lock on the project's primary workspace (not {{@2}}, etc.), which should only be possible if some code has acquired a "quick" lease on the workspace but then failed to ever release it. There are no threads shown holding the workspace lock. Several threads in {{AbstractProject.pollWithWorkspace}} and elsewhere are waiting for it, in some cases for days.

          https://github.com/jenkinsci/jenkins/blob/jenkins-1.580.1/core/src/main/java/hudson/model/AbstractProject.java#L1442-1445 shows a potential window of vulnerability: if {{getBuiltOn}}, {{createLauncher}}, {{decorateByEnv}}, or {{getEnvironment}} threw an exception, and it went reported or was somehow lost in the logs, the lease could be left locked indefinitely.
          A user reports many builds stuck waiting for a lock on the project's primary workspace (not {{@2}}, etc.), which should only be possible if some code has acquired a "quick" lease on the workspace but then failed to ever release it. There are no threads shown holding the workspace lock. Several threads in {{AbstractProject.pollWithWorkspace}} and elsewhere are waiting for it, in some cases for days.

          https://github.com/jenkinsci/jenkins/blob/jenkins-1.580.1/core/src/main/java/hudson/model/AbstractProject.java#L1442-1445 shows a potential window of vulnerability: if {{getBuiltOn}}, {{createLauncher}}, {{decorateByEnv}}, or {{getEnvironment}} threw an exception, and it went unreported or was somehow lost in the logs, the lease could be left locked indefinitely.
          olivergondza Oliver Gond┼ża made changes -
          Labels lock lts-candidate polling robustness workspace 1.596.1-fixed lock polling robustness workspace
          rtyler R. Tyler Croy made changes -
          Workflow JNJira [ 160192 ] JNJira + In-Review [ 196359 ]

            People

            Assignee:
            jglick Jesse Glick
            Reporter:
            jglick Jesse Glick
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: