Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-3580

Workspace deleted when subversion checkout happens

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • subversion-plugin
    • None
    • Platform: All, OS: All

      When a subversion checkout is done, the entire workspace is deleted - not just
      the previous checkout folder. There may be other files held in the workspace
      outside the checkout folder - especially when the same workspace is share by
      multiple jobs. Code should be changed so that only the previous checkout
      folder, if there is one, is deleted before a checkout is done.

          [JENKINS-3580] Workspace deleted when subversion checkout happens

          mdonohue added a comment -

          This changes the behavior of existing Hudson installations - when a user
          upgrades, their workspace will no longer be cleaned if they have the fresh
          checkout option selected for SVN. The functionality associated with the UI
          feature has already flipped and flopped once. Now it's going to flip again.
          This is not a pleasant user experience.

          3966 is nice, but doing all this change at once is surprising for users - I
          think the dev list should be notified of the plan here.

          mdonohue added a comment - This changes the behavior of existing Hudson installations - when a user upgrades, their workspace will no longer be cleaned if they have the fresh checkout option selected for SVN. The functionality associated with the UI feature has already flipped and flopped once. Now it's going to flip again. This is not a pleasant user experience. 3966 is nice, but doing all this change at once is surprising for users - I think the dev list should be notified of the plan here.

          Rolled back from 1.315 release in rev.19544. See
          http://www.nabble.com/attention-all-subversion-users-td24335693.html for the
          discussion.

          Kohsuke Kawaguchi added a comment - Rolled back from 1.315 release in rev.19544. See http://www.nabble.com/attention-all-subversion-users-td24335693.html for the discussion.

          zixenator added a comment -

          I was burned by this again today (using Hudson 1.340). I had to add a new job to Hudson to build a C++ app in the same workspace as 8 other applications (they share a lot of subsystems). When Hudson ran the job, it tried to check out the source from the SVN repository. Here is what it wrote in the log:

          Started by upstream project "Subsystems CI" build number 584
          Checking out a fresh workspace because C:\Hudson\GS CI\Grapher\trunk doesn't exist
          

          It is true that the folder didn't exist - I just added the project and expected Hudson to check it out to the workspace. What I didn't expect was for Hudson to delete the ENTIRE workspace because of this. That's right - it deletes all files for all projects in the shared workspace. This is truly unexpected and unfriendly. It takes hours to check out all that source from the repository. Then I need to manually check out the source for the new folder outside of Hudson. Then Hudson will work. <sigh>

          All in all, Hudson is a great product and the Hudson developers have my sincere thanks. But this one issue has caused me many hours of grief. Please reconsider re-implementing kaxelson's changes. He was on the right track with this - at least for our use case. And if this causes problems for other users, perhaps you can add a configuration option:

          [X] Never delete shared workspace

          zixenator added a comment - I was burned by this again today (using Hudson 1.340). I had to add a new job to Hudson to build a C++ app in the same workspace as 8 other applications (they share a lot of subsystems). When Hudson ran the job, it tried to check out the source from the SVN repository. Here is what it wrote in the log: Started by upstream project "Subsystems CI" build number 584 Checking out a fresh workspace because C:\Hudson\GS CI\Grapher\trunk doesn't exist It is true that the folder didn't exist - I just added the project and expected Hudson to check it out to the workspace. What I didn't expect was for Hudson to delete the ENTIRE workspace because of this. That's right - it deletes all files for all projects in the shared workspace. This is truly unexpected and unfriendly. It takes hours to check out all that source from the repository. Then I need to manually check out the source for the new folder outside of Hudson. Then Hudson will work. <sigh> All in all, Hudson is a great product and the Hudson developers have my sincere thanks. But this one issue has caused me many hours of grief. Please reconsider re-implementing kaxelson's changes. He was on the right track with this - at least for our use case. And if this causes problems for other users, perhaps you can add a configuration option: [X] Never delete shared workspace

          Rosen Diankov added a comment -

          I second with zixenator, when the workspace gets deleted like this, it makes it very difficult to do advanced configurations. I had my checkout option as "svn update as much as possible", but it still deleted the entire workspace when it couldn't find the initial check out. I never even asked for a fresh check out...

          A "Never delete entire workspace" check box would be great to have. There's also another option:

          Currently there's 4 different check-out strategies:

          • svn update as much as possible
          • emulate clean checkout
          • check out fresh copy
          • svn update + svn revert

          Maybe you guys could add a

          • svn update as much as possible, don't delete workspace

          Rosen Diankov added a comment - I second with zixenator, when the workspace gets deleted like this, it makes it very difficult to do advanced configurations. I had my checkout option as "svn update as much as possible", but it still deleted the entire workspace when it couldn't find the initial check out. I never even asked for a fresh check out... A "Never delete entire workspace" check box would be great to have. There's also another option: Currently there's 4 different check-out strategies: svn update as much as possible emulate clean checkout check out fresh copy svn update + svn revert Maybe you guys could add a svn update as much as possible, don't delete workspace

          kutzi added a comment -

          The link to the discussion on the dev list from Kohsuke's comment is dead. This is the new one:
          http://jenkins.361315.n4.nabble.com/attention-all-subversion-users-td393646.html

          kutzi added a comment - The link to the discussion on the dev list from Kohsuke's comment is dead. This is the new one: http://jenkins.361315.n4.nabble.com/attention-all-subversion-users-td393646.html

          We're seeing random deletions of our workspace even without any interaction with SVN. We have a few jobs where we turned off SCM polling and we've seen the workspace mysteriously go bye-bye for those.

          Can anyone explain what might be the cause?

          Daniel Kirkdorffer added a comment - We're seeing random deletions of our workspace even without any interaction with SVN. We have a few jobs where we turned off SCM polling and we've seen the workspace mysteriously go bye-bye for those. Can anyone explain what might be the cause?

          michael elder added a comment -

          I'm constantly snapping our build machine because of this problem. Since the project we are working on has multiple versions, it is more convenient to keep them in a single, custom workspace dir. If for some reason SVN locks one of the project dirs, the next time it updates the entire workspace is rm -rf'ed

          Is there a plugin or a way to sidestep this issue until it is resolved?

          michael elder added a comment - I'm constantly snapping our build machine because of this problem. Since the project we are working on has multiple versions, it is more convenient to keep them in a single, custom workspace dir. If for some reason SVN locks one of the project dirs, the next time it updates the entire workspace is rm -rf'ed Is there a plugin or a way to sidestep this issue until it is resolved?

          Alexandru Gheorghe added a comment - - edited

          Still there with subversion-plugin 2.5, our configuration is pulling many folders under one directory (as it would resemble when the library is built) and it should look like this (assuming we're in $WORKSPACE of the job):

          .
          ├── COPYING
          ├── COPYING.LESSER
          ├── coverage.xml
          ├── src
          │   ├── commands
          │   ├── core
          │   ├── __init__.py
          │   ├── ...
          │   └── webshell
          ├── core_tests
          │   ├── ...
          │   ├── __init__.py
          │   └── test_version.py
          │
          ├──flake8.log
          └── ...
          

          We fetch different folders and put them all in src/ which then we export to PYTHONPATH when we run the unittests from core_tests (with nosetests).

          The problem is that at each fetch from SVN even if we don't want and we specify this to SVN to only update we get the workspace src/ folder erased!

          Is it possible to avoid this? A snippet of the fetch:

          Switching to https://.../core at revision '2015-03-06T12:05:11.289 +0100'
          D         __init__.py
          A         tests
          ...
          

          now comes the next checkout

          Switching to https://.../lib at revision '2015-03-06T12:05:11.289 +0100'
          D         core
          D         data
          D         datatypes
          A         commands
          

          as you can see the "D" will erase previous fetches afterwards adding the new ones. Is this the intended behavior?





          Later addition: after a few more builds it seems it "fixes by itself" and concludes the build successful without any importation failures of any libraries (caused by the then deleted folders), not sure how correct this is how and needs an analysis from our part, but is this normal?

          Alexandru Gheorghe added a comment - - edited Still there with subversion-plugin 2.5, our configuration is pulling many folders under one directory (as it would resemble when the library is built) and it should look like this (assuming we're in $WORKSPACE of the job): . ├── COPYING ├── COPYING.LESSER ├── coverage.xml ├── src │   ├── commands │ ├── core │   ├── __init__.py │ ├── ... │   └── webshell ├── core_tests │ ├── ... │   ├── __init__.py │   └── test_version.py │ ├──flake8.log └── ... We fetch different folders and put them all in src/ which then we export to PYTHONPATH when we run the unittests from core_tests (with nosetests). The problem is that at each fetch from SVN even if we don't want and we specify this to SVN to only update we get the workspace src/ folder erased! Is it possible to avoid this? A snippet of the fetch: Switching to https://.../core at revision '2015-03-06T12:05:11.289 +0100' D __init__.py A tests ... now comes the next checkout Switching to https://.../lib at revision '2015-03-06T12:05:11.289 +0100' D core D data D datatypes A commands as you can see the "D" will erase previous fetches afterwards adding the new ones. Is this the intended behavior? Later addition: after a few more builds it seems it "fixes by itself" and concludes the build successful without any importation failures of any libraries (caused by the then deleted folders), not sure how correct this is how and needs an analysis from our part, but is this normal?

          Daniel Beck added a comment -

          alghe:
          In my experience the modules need to be ordered by increasing root folder depth, e.g. my first module checks out to `.`, and subsequent ones to `foo`, `bar`, etc. – maybe something similar helps in your situation.

          Note that this issue is ancient and probably should not be revived. File new bugs, or (better if you're not sure it's a bug) ask the jenkinsci-users mailing list.

          Daniel Beck added a comment - alghe : In my experience the modules need to be ordered by increasing root folder depth, e.g. my first module checks out to `.`, and subsequent ones to `foo`, `bar`, etc. – maybe something similar helps in your situation. Note that this issue is ancient and probably should not be revived. File new bugs, or (better if you're not sure it's a bug) ask the jenkinsci-users mailing list.

          Vasili Galka added a comment -

          alghe: I agree with danielbeck, what you described does not seem to be related to the initial point of this issue. Also, I don't fully understand what behaviour you expected. Can you please provide a manual list of svn commands that produce your desired behaviour? Then we can analyze how it differs from what Jenkins does.

          Beside alghe comment from 2015, can anyone please explain what is this issue about and why is it still opened? Does the initial problem still exist? I tried digging all the above history, but I fail to find a clear scenario description defining the problem.

          Vasili Galka added a comment - alghe : I agree with danielbeck , what you described does not seem to be related to the initial point of this issue. Also, I don't fully understand what behaviour you expected. Can you please provide a manual list of svn commands that produce your desired behaviour? Then we can analyze how it differs from what Jenkins does. Beside alghe comment from 2015, can anyone please explain what is this issue about and why is it still opened? Does the initial problem still exist? I tried digging all the above history, but I fail to find a clear scenario description defining the problem.

            kaxelson kaxelson
            kaxelson kaxelson
            Votes:
            5 Vote for this issue
            Watchers:
            12 Start watching this issue

              Created:
              Updated:
              Resolved: