Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-10115

Recursive workspace cleanup fails

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • filesystem_scm-plugin
    • None
    • Jenkins running on CentOS 5.x, version 1.412 and 1.417

      In order to reproduce do the following steps:

      • Create a new buildplan (free-style project)
      • Source code managment: select a subversion path (doesn't matter which, just something your can check out)
      • Set the Check-out strategy to 'Always check out a fresh copy'
      • Add a 'execute shell' build script and enter the following commands:
      mkdir -p a/b/c/d/e
      mkdir -p a/b/c/d/f
      touch a/b/c/d/e/1
      touch a/b/c/d/e/2
      touch a/b/c/d/f/1
      touch a/b/c/d/f/2
      ln -s ../../../../../a/b/c/d/f/1 a/b/c/d/e/4
      ln -s ../../../../../a/b/c/d/f/2 a/b/c/d/e/5
      ln -s ../../../../../a/b/c/d/e/1 a/b/c/d/f/4
      ln -s ../../../../../a/b/c/d/e/2 a/b/c/d/f/5
      chmod -R 555 a
      chmod 444 a/b/c/d/e/*
      chmod 444 a/b/c/d/f/*
      

      Now if you launch this job, this results in the following pattern: success, fail, success, fail, ... the reason for this pattern does not immediately make sense

      Where a fail contains the following log:

      Started by user anonymous
      Building on master
      Cleaning workspace /home/hudson/.hudson/jobs/edb-bla/workspace
      java.io.IOException: Unable to delete /home/hudson/.hudson/jobs/edb-bla/workspace/a/b/c/d/e - files in dir: [/home/hudson/.hudson/jobs/edb-bla/workspace/a/b/c/d/e/5]
      	at hudson.Util.deleteFile(Util.java:262)
      	at hudson.Util.deleteRecursive(Util.java:305)
      	at hudson.Util.deleteContentsRecursive(Util.java:224)
      	at hudson.Util.deleteRecursive(Util.java:304)
      	at hudson.Util.deleteContentsRecursive(Util.java:224)
      	at hudson.Util.deleteRecursive(Util.java:304)
      	at hudson.Util.deleteContentsRecursive(Util.java:224)
      	at hudson.Util.deleteRecursive(Util.java:304)
      	at hudson.Util.deleteContentsRecursive(Util.java:224)
      	at hudson.Util.deleteRecursive(Util.java:304)
      	at hudson.Util.deleteContentsRecursive(Util.java:224)
      	at hudson.scm.subversion.CheckoutUpdater$1.perform(CheckoutUpdater.java:67)
      	at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:135)
      	at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:726)
      	at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:707)
      	at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:691)
      	at hudson.FilePath.act(FilePath.java:758)
      	at hudson.FilePath.act(FilePath.java:740)
      	at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:684)
      	at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:633)
      	at hudson.model.AbstractProject.checkout(AbstractProject.java:1182)
      	at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:537)
      	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:425)
      	at hudson.model.Run.run(Run.java:1376)
      	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
      	at hudson.model.ResourceController.execute(ResourceController.java:88)
      	at hudson.model.Executor.run(Executor.java:146)
      Finished: FAILURE
      

      This happens both on builds running locally as builds executed on a buildslave. Since this sometimes succeeds, this is not linked with a unix permission issue. Also, if the symlinks do not point to valid destinations then this doesn't change the story either.

      Now after a successful run the path looks like (excluding the svn checkout):

      [hudson@mellon workspace]$ ls -Ral a
      a:
      total 24
      dr-xr-xr-x 3 hudson hudson 4096 Jun 27 14:27 .
      drwxrwxr-x 4 hudson hudson 4096 Jun 27 14:27 ..
      dr-xr-xr-x 3 hudson hudson 4096 Jun 27 14:27 b
      
      a/b:
      total 24
      dr-xr-xr-x 3 hudson hudson 4096 Jun 27 14:27 .
      dr-xr-xr-x 3 hudson hudson 4096 Jun 27 14:27 ..
      dr-xr-xr-x 3 hudson hudson 4096 Jun 27 14:27 c
      
      a/b/c:
      total 24
      dr-xr-xr-x 3 hudson hudson 4096 Jun 27 14:27 .
      dr-xr-xr-x 3 hudson hudson 4096 Jun 27 14:27 ..
      dr-xr-xr-x 4 hudson hudson 4096 Jun 27 14:27 d
      
      a/b/c/d:
      total 32
      dr-xr-xr-x 4 hudson hudson 4096 Jun 27 14:27 .
      dr-xr-xr-x 3 hudson hudson 4096 Jun 27 14:27 ..
      dr-xr-xr-x 2 hudson hudson 4096 Jun 27 14:27 e
      dr-xr-xr-x 2 hudson hudson 4096 Jun 27 14:27 f
      
      a/b/c/d/e:
      total 32
      dr-xr-xr-x 2 hudson hudson 4096 Jun 27 14:27 .
      dr-xr-xr-x 4 hudson hudson 4096 Jun 27 14:27 ..
      -r--r--r-- 1 hudson hudson    0 Jun 27 14:27 1
      -r--r--r-- 1 hudson hudson    0 Jun 27 14:27 2
      lrwxrwxrwx 1 hudson hudson   26 Jun 27 14:27 4 -> ../../../../../a/b/c/d/f/1
      lrwxrwxrwx 1 hudson hudson   26 Jun 27 14:27 5 -> ../../../../../a/b/c/d/f/2
      
      a/b/c/d/f:
      total 32
      dr-xr-xr-x 2 hudson hudson 4096 Jun 27 14:27 .
      dr-xr-xr-x 4 hudson hudson 4096 Jun 27 14:27 ..
      -r--r--r-- 1 hudson hudson    0 Jun 27 14:27 1
      -r--r--r-- 1 hudson hudson    0 Jun 27 14:27 2
      lrwxrwxrwx 1 hudson hudson   26 Jun 27 14:27 4 -> ../../../../../a/b/c/d/e/1
      lrwxrwxrwx 1 hudson hudson   26 Jun 27 14:27 5 -> ../../../../../a/b/c/d/e/2
      

      After the failed build, it looks like this on the os:

      [hudson@mellon workspace]$ ls -Ral a
      a:
      total 24
      dr-xr-xr-x 3 hudson hudson 4096 Jun 27 14:27 .
      drwxrwxr-x 4 hudson hudson 4096 Jun 27 14:27 ..
      dr-xr-xr-x 3 hudson hudson 4096 Jun 27 14:27 b
      
      a/b:
      total 24
      dr-xr-xr-x 3 hudson hudson 4096 Jun 27 14:27 .
      dr-xr-xr-x 3 hudson hudson 4096 Jun 27 14:27 ..
      dr-xr-xr-x 3 hudson hudson 4096 Jun 27 14:27 c
      
      a/b/c:
      total 24
      dr-xr-xr-x 3 hudson hudson 4096 Jun 27 14:27 .
      dr-xr-xr-x 3 hudson hudson 4096 Jun 27 14:27 ..
      drwxr-xr-x 3 hudson hudson 4096 Jun 27 14:29 d
      
      a/b/c/d:
      total 24
      drwxr-xr-x 3 hudson hudson 4096 Jun 27 14:29 .
      dr-xr-xr-x 3 hudson hudson 4096 Jun 27 14:27 ..
      drwxr-xr-x 2 hudson hudson 4096 Jun 27 14:29 e
      
      a/b/c/d/e:
      total 20
      drwxr-xr-x 2 hudson hudson 4096 Jun 27 14:29 .
      drwxr-xr-x 3 hudson hudson 4096 Jun 27 14:29 ..
      lrwxrwxrwx 1 hudson hudson   26 Jun 27 14:27 5 -> ../../../../../a/b/c/d/f/2
      

      So it managed to clean 'something' and this something is enough to make it succeed the next time.

      (Note, while experimenting, I've also been able to create success, fail, fail, success, fail, fail, ... patterns, I've also seen this happen when the symlinks are not valid something (e.g. omit the ../../../../'s) but since on one of my projects it failed with a symlink I introduced this in my testcase).

            samngms samngms
            eliedebrauwer Elie De Brauwer
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: