Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-17228

Fail build when subversion workspace is locked

      This is a request to backport a Hudson fix into Jenkins. This is a 2-line change done in Hudson Subversion plugin 2.3.3, SHA 37ca9c134702878e6bb824dde108d63c6df043d8

      https://github.com/hudson-plugins/subversion-plugin/commit/37ca9c134702878e6bb824dde108d63c6df043d8

      Rationale: subversion workspace often remains locked if previous svn update failed because of bad network connection, wiping the workspace is not useful for anything in this situation, rather the opposite.

          [JENKINS-17228] Fail build when subversion workspace is locked

          jan_ruzicka added a comment -

          current location of the fixed section in
          jenkinsci/subversion-plugin
          is
          src/main/java/hudson/scm/subversion/UpdateUpdater.java
          lines
          181 and 182

          jan_ruzicka added a comment - current location of the fixed section in jenkinsci/subversion-plugin is src/main/java/hudson/scm/subversion/UpdateUpdater.java lines 181 and 182

          jan_ruzicka added a comment -

          Will the failing build unlock the workspace?
          or will all the fallowing builds fail?

          jan_ruzicka added a comment - Will the failing build unlock the workspace? or will all the fallowing builds fail?

          Paavo Helde added a comment -

          In principle the workspace might also be locked because somebody else has logged into the slave computer and doing some svn operations manually. Therefore I would not suggest automatic "svn cleanup" or something like that, just failing the build (as the Hudson fix does) would be fine IMO. A user can always wipe the workspace explicitly if he/she is not able to solve the problem otherwise.

          Paavo Helde added a comment - In principle the workspace might also be locked because somebody else has logged into the slave computer and doing some svn operations manually. Therefore I would not suggest automatic "svn cleanup" or something like that, just failing the build (as the Hudson fix does) would be fine IMO. A user can always wipe the workspace explicitly if he/she is not able to solve the problem otherwise.

          John Elliot added a comment -

          I'd strongly request this fix too.

          We have very large repositories and a full rebuild takes many hours. I would much rather have a chance to fix a repository manually than have jenkins go away and delete the folder.

          The build should just keep failing until the lock is fixed manually.

          John Elliot added a comment - I'd strongly request this fix too. We have very large repositories and a full rebuild takes many hours. I would much rather have a chance to fix a repository manually than have jenkins go away and delete the folder. The build should just keep failing until the lock is fixed manually.

          jan_ruzicka added a comment -

          jan_ruzicka added a comment - the Hudson commit moved around https://github.com/hudson3-plugins/subversion-plugin/commit/37ca9c134702878e6bb824dde108d63c6df043d8

          david etkins added a comment -

          this is not an improvement this is a bug, and it should be fixed ASAP

          david etkins added a comment - this is not an improvement this is a bug, and it should be fixed ASAP

          Greg Roper added a comment -

          Any plans to get this scheduled? Maybe give the user the option to check out a new workspace if the workspace is locked rather than defaulting to it? I'd much rather my build to fail, I can clean up the workspace myself in a few minutes and not have to wait hours for fresh workspace to be built.

          Greg Roper added a comment - Any plans to get this scheduled? Maybe give the user the option to check out a new workspace if the workspace is locked rather than defaulting to it? I'd much rather my build to fail, I can clean up the workspace myself in a few minutes and not have to wait hours for fresh workspace to be built.

          Tim Bradt added a comment -

          Perhaps I'm not understanding this issue correctly. It seems to me that you want the Jenkins job/build to immediately fail if the subversion workspace is locked. I'm confused as to why this is an open issue as that is exactly the behavior I'm seeing with Subversion Plug-in v2.5. For me, this is undesirable due to the structure of our repository. I have multiple builds trying to update their individual portions of the working copy. Unfortunately, svn locks the working copy one directory above the one I'm trying to update, so I get failures if multiple builds are triggered concurrently, even though they are in different sections of the repository. The way it currently works, I'm notifying commiters that they have a build failure even though the changes themselves did not cause the failure, and then if they have nothing more to commit there is no choice but to MANUALLY trigger the job again. Blech.

          I would prefer to have this be configurable. Let me decide if I want it to fail (as it sounds like is the desire here), or try again every ___ seconds with ___ total attempts. This would solve my problem without the other alternatives that I am currently limited to, all of which are fairly drastic and complex.

          So far I'm not finding another issue with that request so will probably submit it.

          Tim Bradt added a comment - Perhaps I'm not understanding this issue correctly. It seems to me that you want the Jenkins job/build to immediately fail if the subversion workspace is locked. I'm confused as to why this is an open issue as that is exactly the behavior I'm seeing with Subversion Plug-in v2.5. For me, this is undesirable due to the structure of our repository. I have multiple builds trying to update their individual portions of the working copy. Unfortunately, svn locks the working copy one directory above the one I'm trying to update, so I get failures if multiple builds are triggered concurrently, even though they are in different sections of the repository. The way it currently works, I'm notifying commiters that they have a build failure even though the changes themselves did not cause the failure, and then if they have nothing more to commit there is no choice but to MANUALLY trigger the job again. Blech. I would prefer to have this be configurable. Let me decide if I want it to fail (as it sounds like is the desire here), or try again every ___ seconds with ___ total attempts. This would solve my problem without the other alternatives that I am currently limited to, all of which are fairly drastic and complex. So far I'm not finding another issue with that request so will probably submit it.

          Paavo Helde added a comment -

          This issue is about when Jenkins has checked out and updating the working copy by itself. In this scenario every job should get its own workspace so concurrent access to it should not happen AFAIK. If there is concurrent access this probably means the svn checkout was not done by Jenkins; that would also explain why it does not attempt to wipe the workspace. Or have I misunderstood something here?

          Running concurrent builds in the same checkout folder seems not so good idea anyway. Jenkins provides an easy way to avoid that by setting the allowed number of executors to 1.

          Paavo Helde added a comment - This issue is about when Jenkins has checked out and updating the working copy by itself. In this scenario every job should get its own workspace so concurrent access to it should not happen AFAIK. If there is concurrent access this probably means the svn checkout was not done by Jenkins; that would also explain why it does not attempt to wipe the workspace. Or have I misunderstood something here? Running concurrent builds in the same checkout folder seems not so good idea anyway. Jenkins provides an easy way to avoid that by setting the allowed number of executors to 1.

          Daniel Beck added a comment -

          Tim: Sharing workspaces between jobs is essentially an anti-pattern. That said, there are several plugins allowing you to limit parallel execution of jobs, e.g. Throttle Concurrent Builds Plugin, which will prevent this issue in your situation.

          Daniel Beck added a comment - Tim: Sharing workspaces between jobs is essentially an anti-pattern. That said, there are several plugins allowing you to limit parallel execution of jobs, e.g. Throttle Concurrent Builds Plugin, which will prevent this issue in your situation.

          Mike Konikoff added a comment - - edited

          I'm using jenkins to auto-update my eclipse project directories, and subclipse in the IDE to commit/log/etc. and occasionally manually update my workspace. This saves me a ton of time on synchronizing and avoiding conflicts by integrating my working copy early and often. Without this fix, any interruption of subclipse that leaves the workspace locked and not cleaned up before next build, results in jenkins wiping out uncommitted changes in my working copy. Although it doesn't happen often, this wastes a lot of the time saved using CI to begin with, on restoring workspace and last set of changes.

          Mike Konikoff added a comment - - edited I'm using jenkins to auto-update my eclipse project directories, and subclipse in the IDE to commit/log/etc. and occasionally manually update my workspace. This saves me a ton of time on synchronizing and avoiding conflicts by integrating my working copy early and often. Without this fix, any interruption of subclipse that leaves the workspace locked and not cleaned up before next build, results in jenkins wiping out uncommitted changes in my working copy. Although it doesn't happen often, this wastes a lot of the time saved using CI to begin with, on restoring workspace and last set of changes.

          Frank Hickman added a comment -

          I would dearly love an option to fail the build, instead of automatically checking out a fresh workspace. Let me explain the use case, which has nothing to do with shared workspaces:

          We've got a CI job that runs for our huge game development project. Because of all the large binary assets involved, checking out a fresh workspace (from a remove SVN repository) can take about 6 hours. Our ISP is not great either, so frequently a long SVN operation may fail due to some network error.

          At present, every time that happens, the next run of the job decides that 'Workspace appear to be locked, so getting a fresh workspace'. This means we have to wait another 6 hours for the job to finish. We'd much rather have the option to fail the build, and attend to the SVN working copy manually.

          Thank you!

          Frank Hickman added a comment - I would dearly love an option to fail the build, instead of automatically checking out a fresh workspace. Let me explain the use case, which has nothing to do with shared workspaces: We've got a CI job that runs for our huge game development project. Because of all the large binary assets involved, checking out a fresh workspace (from a remove SVN repository) can take about 6 hours . Our ISP is not great either, so frequently a long SVN operation may fail due to some network error. At present, every time that happens, the next run of the job decides that 'Workspace appear to be locked, so getting a fresh workspace'. This means we have to wait another 6 hours for the job to finish. We'd much rather have the option to fail the build, and attend to the SVN working copy manually. Thank you!

          Cheers
          We have the same problem in version 2.5.
          "Workspace appear to be locked, so getting a fresh workspace
          Cleaning local Directory ."

          When a build fails, the next one makes a fresh checkout.
          Our repositories are very large and it takes too long. We also see that while checkout is performed jenkins consumes too much cpu.

          Miguel Angel Hernandez added a comment - Cheers We have the same problem in version 2.5. "Workspace appear to be locked, so getting a fresh workspace Cleaning local Directory ." When a build fails, the next one makes a fresh checkout. Our repositories are very large and it takes too long. We also see that while checkout is performed jenkins consumes too much cpu.

          Alexis Bietti added a comment -

          Are there any alternatives to the factory Subversion plugin? Seeing how long this issue has been ignored is kind of sad actually. Cheers

          Alexis Bietti added a comment - Are there any alternatives to the factory Subversion plugin? Seeing how long this issue has been ignored is kind of sad actually. Cheers

          John Dahl added a comment -

          Seeing this still unsolved hurts, especially when we are building repositories of gigabyte size. The app suddenly goes down for 10+ mins during refreshing. Do we have any temporary workaround until this is fixed?

          John Dahl added a comment - Seeing this still unsolved hurts, especially when we are building repositories of gigabyte size. The app suddenly goes down for 10+ mins during refreshing. Do we have any temporary workaround until this is fixed?

          Paavo Helde added a comment -

          We are currently using the "Install Hudson" workaround ...

          Paavo Helde added a comment - We are currently using the "Install Hudson" workaround ...

          Ruben Stein added a comment -

          The link to the fix is broken, if someone could point to these two lines again, one could try to rebuild a patched version.

          Ruben Stein added a comment - The link to the fix is broken, if someone could point to these two lines again, one could try to rebuild a patched version.

          Daniel Beck added a comment -

          if someone could point to these two lines again

          rst It took me a minute to find the new repo location. You can do it if you try!

          Daniel Beck added a comment - if someone could point to these two lines again rst It took me a minute to find the new repo location. You can do it if you try!

          John Dahl added a comment -

          John Dahl added a comment - Please try this link https://github.com/hudson3-plugins/subversion-plugin/commit/37ca9c134702878e6bb824dde108d63c6df043d8

          John Dahl added a comment -

          We are also trying Hudson at the moment, but we are lacking many plugins that are available in Jenkins. Plugins available in Hudson have also stopped updating

          John Dahl added a comment - We are also trying Hudson at the moment, but we are lacking many plugins that are available in Jenkins. Plugins available in Hudson have also stopped updating

          Nick Owens added a comment -

          Please fix this or at least add a toggle to svn settings so you can choose to either "just fail", or "get fresh workspace" if the error is SVNErrorCode.WC_LOCKED

          Nick Owens added a comment - Please fix this or at least add a toggle to svn settings so you can choose to either "just fail", or "get fresh workspace" if the error is SVNErrorCode.WC_LOCKED

          Petr Kubat added a comment -

          Hello, I cant believe nobody touched such an issue for over 4 years. This is really a problem. Is there any official statement from Jenkins maintainers? For example clearly stating that they decided to ignore this issue?

          Petr Kubat added a comment - Hello, I cant believe nobody touched such an issue for over 4 years. This is really a problem. Is there any official statement from Jenkins maintainers? For example clearly stating that they decided to ignore this issue?

          Petr Kubat added a comment -

          Ok, I created a pull request making the same change as was done in Hudson. Let's see what happens.

          Petr Kubat added a comment - Ok, I created a pull request making the same change as was done in Hudson. Let's see what happens.

          Just had this blow up a whole series of builds this morning.  Don't see a good reason why the pull request was rejected.

          Has anyone found a reasonable workaround?  Best I can figure out is to try to checkout in a sub-folder that is safer to have blasted away then go back up to my parent.

          Michael Thompson added a comment - Just had this blow up a whole series of builds this morning.  Don't see a good reason why the pull request was rejected. Has anyone found a reasonable workaround?  Best I can figure out is to try to checkout in a sub-folder that is safer to have blasted away then go back up to my parent.

          Please make it an option to not get a fresh workspace. I have to write custom svn bash and bat files to manually update svn just to prevent this from happening as it takes two hours to checkout our svn and its unacceptable.

          Devon Scott-Tunkin added a comment - Please make it an option to not get a fresh workspace. I have to write custom svn bash and bat files to manually update svn just to prevent this from happening as it takes two hours to checkout our svn and its unacceptable.

            Unassigned Unassigned
            paavo256 Paavo Helde
            Votes:
            20 Vote for this issue
            Watchers:
            21 Start watching this issue

              Created:
              Updated: