Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-21369

Git client fails to remove files/directories with "git clean -fdx"

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Critical
    • Resolution: Fixed
    • Component/s: git-client-plugin
    • Labels:
      None
    • Environment:
      OS: Windows 2008 R2
      Java: 1.6.0_26
      Jenkins: 1.547
      Git client: Newest (1.6.1)
      Other plugins: All are updated, like Git (2.0.1)

      Jenkins is running as Windows service, with Local system account
    • Similar Issues:

      Description

      This all has worked fine previously, but latest updates for some reason there have been these kind of issues, as described below.

      1. Start Jenkins service
      2. Trigger ex. 20 builds
      3. Wait and see if one of them fails to build
      4. See if the reason is Jenkins issue, not msbuild/unit test
      5. Rest of builds that starts after failed one, will fail also with same output as below
      6. Wipe workspaces manually
      7. Builds will most likely start working again

      Triggered by Gerrit: http://xxx002:8080/4195
      [EnvInject] - Loading node environment variables.
      Building on master in workspace r:\jenkins-workspaces
      Fetching changes from the remote Git repository
      Fetching upstream changes from ssh://hudson@xxxx002:29418/xxx.git
      Checking out Revision 45d62ed078309607e0b6a24b842ff33c93f3b740 (feature/WI3076)
      Cleaning workspace
      Resetting working tree
      FATAL: Command "git clean -fdx" returned status code 1:
      stdout: Removing Source/xxx/zzz/Areas
      Removing Source/xxx/zzz/Content
      Removing Source/xxx/zzz/Scripts
      Removing Source/xxx/zzz/Views

      stderr: warning: failed to remove Source/xxx/zzz/Areas
      warning: failed to remove Source/xxx/zzz/Content
      warning: failed to remove Source/xxx/zzz/Scripts
      warning: failed to remove Source/xxx/zzz/Views

      hudson.plugins.git.GitException: Command "git clean -fdx" returned status code 1:
      stdout: Removing Source/xxx/zzz/Areas
      Removing Source/xxx/zzz/Content
      Removing Source/xxx/zzz/Scripts
      Removing Source/xxx/zzz/Views

      stderr: warning: failed to remove Source/xxx/zzz/Areas
      warning: failed to remove Source/xxx/zzz/Content
      warning: failed to remove Source/xxx/zzz/Scripts
      warning: failed to remove Source/xxx/zzz/Views

      at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1086)
      at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1063)
      at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:900)
      at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:910)
      at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.clean(CliGitAPIImpl.java:367)
      at hudson.plugins.git.GitAPI.clean(GitAPI.java:251)
      at hudson.plugins.git.extensions.impl.CleanCheckout.onCheckoutCompleted(CleanCheckout.java:28)
      at hudson.plugins.git.GitSCM.checkout(GitSCM.java:890)
      at hudson.model.AbstractProject.checkout(AbstractProject.java:1415)
      at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:652)
      at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88)
      at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:561)
      at hudson.model.Run.execute(Run.java:1678)
      at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
      at hudson.model.ResourceController.execute(ResourceController.java:88)
      at hudson.model.Executor.run(Executor.java:231)

        Attachments

          Activity

          Hide
          tomate Thomas COAN added a comment -

          Thanks Mark,
          I have tryed to run the job after a reboot => no old process holding the folder => same problem !
          in batch mode (del /F/Q [...]\target , it fails => no error, but no delete!)

          However I have investigated on the commit that generates the problem, and I have found a new file that generates a path greater than 255 characters in my target folder.
          Could it be the root cause ?

          I will check with the dev if he can change it and keep you in touch

          I cannot switch on a unix machine (not owned by myself !)

          Show
          tomate Thomas COAN added a comment - Thanks Mark, I have tryed to run the job after a reboot => no old process holding the folder => same problem ! in batch mode (del /F/Q [...] \target , it fails => no error, but no delete!) However I have investigated on the commit that generates the problem, and I have found a new file that generates a path greater than 255 characters in my target folder. Could it be the root cause ? I will check with the dev if he can change it and keep you in touch I cannot switch on a unix machine (not owned by myself !)
          Hide
          danielbeck Daniel Beck added a comment -

          Issue report needs to show that this is a problem with Jenkins. The Sysinternals tools (e.g. Process Explorer) could help determining which process is holding a file handle.

          Show
          danielbeck Daniel Beck added a comment - Issue report needs to show that this is a problem with Jenkins. The Sysinternals tools (e.g. Process Explorer) could help determining which process is holding a file handle.
          Hide
          danielbeck Daniel Beck added a comment -

          FWIW I've repeatedly had to remind my users to log out of the build server instead of just closing the RDC session – make sure there are no sessions with e.g. a cmd.exe in that directory open.

          Show
          danielbeck Daniel Beck added a comment - FWIW I've repeatedly had to remind my users to log out of the build server instead of just closing the RDC session – make sure there are no sessions with e.g. a cmd.exe in that directory open.
          Hide
          danielbeck Daniel Beck added a comment -

          Sorry, didn't see the comment posted just before mine. It's likely that the long path is the problem in which case this is Not a Defect in Jenkins.

          A few suggestions if you're right around ~260 chars and your developer is being stubborn:

          • Use a short Jenkins (master/slave) root directory, e.g. D:\J
          • Change the Workspace Root Directory on the master completely (Configure System; Advanced), e.g. D:\WS if building on the master
          • Set the hudson.model.Slave.workspaceRoot system property to something shorter than workspace if building on a slave
          • Set the hudson.matrix.MatrixConfiguration.useShortWorkspaceName system property to true for any Multi-configuration projects
          • Configure a custom workspace in Advanced Project Options
          Show
          danielbeck Daniel Beck added a comment - Sorry, didn't see the comment posted just before mine. It's likely that the long path is the problem in which case this is Not a Defect in Jenkins. A few suggestions if you're right around ~260 chars and your developer is being stubborn: Use a short Jenkins (master/slave) root directory, e.g. D:\J Change the Workspace Root Directory on the master completely (Configure System; Advanced), e.g. D:\WS if building on the master Set the hudson.model.Slave.workspaceRoot system property to something shorter than workspace if building on a slave Set the hudson.matrix.MatrixConfiguration.useShortWorkspaceName system property to true for any Multi-configuration projects Configure a custom workspace in Advanced Project Options
          Hide
          tomate Thomas COAN added a comment - - edited

          Ok problem comes from the length of windows path > 255.
          Problem is solved for us => should be a jenkins restriction

          thanks Daniel,
          I will check your proposition but we have some restrictions as some jobs are generated by an automate (=> to do in our side for the choice of folder names)

          Show
          tomate Thomas COAN added a comment - - edited Ok problem comes from the length of windows path > 255. Problem is solved for us => should be a jenkins restriction thanks Daniel, I will check your proposition but we have some restrictions as some jobs are generated by an automate (=> to do in our side for the choice of folder names)

            People

            Assignee:
            ndeloof Nicolas De Loof
            Reporter:
            hannupiki Hannu Piki
            Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: