Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-10615

Workspaces seem to be removed prematurely on concurrent jobs

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Critical Critical
    • core

      Concurrent Builds workspaces are sometimes removed before the build steps have completed.
      This is not seen on the primary workspace, only those that get "@2" etc appended.

      This causes a failure on the job in question - symptoms somewhat varying depending on the process running. Generally a file not found, followed by a process complaining that its Current Working directory has gone.

      Hard to reproduce, but reproducibility seems to increase with uptime from rare to more than daily.

      Affects builds running on both linux and windows slave nodes. Not seen on builds running on master.
      Disk space, memory and cpu availability more than adequate.

          [JENKINS-10615] Workspaces seem to be removed prematurely on concurrent jobs

          Code changed in jenkins
          User: Kohsuke Kawaguchi
          Path:
          test/src/test/groovy/hudson/model/AbstractProjectTest.groovy
          http://jenkins-ci.org/commit/jenkins/61cf2df0660e507fab20442f44e917d75f946917
          Log:
          JENKINS-10615

          Reproduced the problem. Workspace is getting released prematurely, before publishers run. This has all sorts of serious problems.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: test/src/test/groovy/hudson/model/AbstractProjectTest.groovy http://jenkins-ci.org/commit/jenkins/61cf2df0660e507fab20442f44e917d75f946917 Log: JENKINS-10615 Reproduced the problem. Workspace is getting released prematurely, before publishers run. This has all sorts of serious problems.

          Code changed in jenkins
          User: Kohsuke Kawaguchi
          Path:
          changelog.html
          core/src/main/java/hudson/model/AbstractBuild.java
          http://jenkins-ci.org/commit/jenkins/d183345007ecd6fae01565975fb48ddec6e47af4
          Log:
          [FIXED JENKINS-10615]

          Hold on to the lease until the very end.
          Previously, the lease was only held until the main build section is over, before publishers start running.

          Compare: https://github.com/jenkinsci/jenkins/compare/7648a95a7c57...d183345007ec

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: changelog.html core/src/main/java/hudson/model/AbstractBuild.java http://jenkins-ci.org/commit/jenkins/d183345007ecd6fae01565975fb48ddec6e47af4 Log: [FIXED JENKINS-10615] Hold on to the lease until the very end. Previously, the lease was only held until the main build section is over, before publishers start running. Compare: https://github.com/jenkinsci/jenkins/compare/7648a95a7c57...d183345007ec

          The root cause of the problem is that the workspace used for a build was released after the main build section is over, so the publishers end up running with the workspace that it no longer owns.

          As a result, workspace clean up thread might try to recover this "no longer in use" workspace, or another build might try to use it, resulting in various failure modes.

          Kohsuke Kawaguchi added a comment - The root cause of the problem is that the workspace used for a build was released after the main build section is over, so the publishers end up running with the workspace that it no longer owns. As a result, workspace clean up thread might try to recover this "no longer in use" workspace, or another build might try to use it, resulting in various failure modes.

          dogfood added a comment -

          Integrated in jenkins_main_trunk #3036

          Result = SUCCESS

          dogfood added a comment - Integrated in jenkins_main_trunk #3036 Result = SUCCESS

          Code changed in jenkins
          User: Kohsuke Kawaguchi
          Path:
          test/src/test/groovy/hudson/model/AbstractProjectTest.groovy
          http://jenkins-ci.org/commit/jenkins/8c8102efd4fa324d34dcb9dc156c37aacb1feb1a
          Log:
          JENKINS-10615

          Reproduced the problem. Workspace is getting released prematurely, before publishers run. This has all sorts of serious problems.

          (cherry picked from commit 61cf2df0660e507fab20442f44e917d75f946917)

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: test/src/test/groovy/hudson/model/AbstractProjectTest.groovy http://jenkins-ci.org/commit/jenkins/8c8102efd4fa324d34dcb9dc156c37aacb1feb1a Log: JENKINS-10615 Reproduced the problem. Workspace is getting released prematurely, before publishers run. This has all sorts of serious problems. (cherry picked from commit 61cf2df0660e507fab20442f44e917d75f946917)

          Code changed in jenkins
          User: Kohsuke Kawaguchi
          Path:
          core/src/main/java/hudson/model/AbstractBuild.java
          http://jenkins-ci.org/commit/jenkins/d95e16428c47933a6a81c5508f92d977ab2efa4d
          Log:
          [FIXED JENKINS-10615]

          Hold on to the lease until the very end.
          Previously, the lease was only held until the main build section is over, before publishers start running.

          (cherry picked from commit d183345007ecd6fae01565975fb48ddec6e47af4)

          Conflicts:
          changelog.html

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: core/src/main/java/hudson/model/AbstractBuild.java http://jenkins-ci.org/commit/jenkins/d95e16428c47933a6a81c5508f92d977ab2efa4d Log: [FIXED JENKINS-10615] Hold on to the lease until the very end. Previously, the lease was only held until the main build section is over, before publishers start running. (cherry picked from commit d183345007ecd6fae01565975fb48ddec6e47af4) Conflicts: changelog.html

          Code changed in jenkins
          User: Jesse Glick
          Path:
          test/src/test/groovy/hudson/model/AbstractProjectTest.groovy
          http://jenkins-ci.org/commit/jenkins/85e9e126773c0bb20a8529a2e6591dde17d7e209
          Log:
          JENKINS-10615 AbstractProjectTest.testWorkspaceLock frequently fails on jenkins.ci due to InterruptedException in HudsonTestCase.setUp.
          Possibly because it is sorted after JENKINS-15156 testGetBuildAfterGC and the test suite times out.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: test/src/test/groovy/hudson/model/AbstractProjectTest.groovy http://jenkins-ci.org/commit/jenkins/85e9e126773c0bb20a8529a2e6591dde17d7e209 Log: JENKINS-10615 AbstractProjectTest.testWorkspaceLock frequently fails on jenkins.ci due to InterruptedException in HudsonTestCase.setUp. Possibly because it is sorted after JENKINS-15156 testGetBuildAfterGC and the test suite times out.

          dogfood added a comment -

          Integrated in jenkins_main_trunk #3111
          JENKINS-10615 AbstractProjectTest.testWorkspaceLock frequently fails on jenkins.ci due to InterruptedException in HudsonTestCase.setUp. (Revision 85e9e126773c0bb20a8529a2e6591dde17d7e209)

          Result = SUCCESS
          Jesse Glick : 85e9e126773c0bb20a8529a2e6591dde17d7e209
          Files :

          • test/src/test/groovy/hudson/model/AbstractProjectTest.groovy

          dogfood added a comment - Integrated in jenkins_main_trunk #3111 JENKINS-10615 AbstractProjectTest.testWorkspaceLock frequently fails on jenkins.ci due to InterruptedException in HudsonTestCase.setUp. (Revision 85e9e126773c0bb20a8529a2e6591dde17d7e209) Result = SUCCESS Jesse Glick : 85e9e126773c0bb20a8529a2e6591dde17d7e209 Files : test/src/test/groovy/hudson/model/AbstractProjectTest.groovy

          Code changed in jenkins
          User: Jesse Glick
          Path:
          test/src/test/groovy/hudson/model/AbstractProjectTest.groovy
          http://jenkins-ci.org/commit/jenkins/389a565de417170f586830ee9fa7a7ec9749fc68
          Log:
          JENKINS-10615 AbstractProjectTest.testWorkspaceLock frequently fails on jenkins.ci due to InterruptedException in HudsonTestCase.setUp.
          Possibly because it is sorted after JENKINS-15156 testGetBuildAfterGC and the test suite times out.

          (cherry picked from commit 85e9e126773c0bb20a8529a2e6591dde17d7e209)

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: test/src/test/groovy/hudson/model/AbstractProjectTest.groovy http://jenkins-ci.org/commit/jenkins/389a565de417170f586830ee9fa7a7ec9749fc68 Log: JENKINS-10615 AbstractProjectTest.testWorkspaceLock frequently fails on jenkins.ci due to InterruptedException in HudsonTestCase.setUp. Possibly because it is sorted after JENKINS-15156 testGetBuildAfterGC and the test suite times out. (cherry picked from commit 85e9e126773c0bb20a8529a2e6591dde17d7e209)

          dogfood added a comment -

          Integrated in jenkins_main_trunk #3715
          JENKINS-10615 AbstractProjectTest.testWorkspaceLock frequently fails on jenkins.ci due to InterruptedException in HudsonTestCase.setUp. (Revision 389a565de417170f586830ee9fa7a7ec9749fc68)

          Result = SUCCESS
          Jesse Glick : 389a565de417170f586830ee9fa7a7ec9749fc68
          Files :

          • test/src/test/groovy/hudson/model/AbstractProjectTest.groovy

          dogfood added a comment - Integrated in jenkins_main_trunk #3715 JENKINS-10615 AbstractProjectTest.testWorkspaceLock frequently fails on jenkins.ci due to InterruptedException in HudsonTestCase.setUp. (Revision 389a565de417170f586830ee9fa7a7ec9749fc68) Result = SUCCESS Jesse Glick : 389a565de417170f586830ee9fa7a7ec9749fc68 Files : test/src/test/groovy/hudson/model/AbstractProjectTest.groovy

            kohsuke Kohsuke Kawaguchi
            dannystaple Danny Staple
            Votes:
            6 Vote for this issue
            Watchers:
            16 Start watching this issue

              Created:
              Updated:
              Resolved: