Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-26660

Add another '-f' to git clean to remove deleted submodules or other repos checked out in workspace in pipeline

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • None
    • Jenkins and multi SCM plugin configured with git repository using submodules
      Alternatively: A pipeline that checks out a second repository in a sub-dir
    • git-client-plugin 3.0.0, git-plugin 4.1.0

      Submodules clean up

      Jenkins SCM plugin can handle git repositories with submodules except for clean up.

      When "Clean before checkout" is selected, it runs 'git reset --hard && git clean -dxf' on the repository and submodules if "Recursively update submodules" is selected. This works for most stuff but it does not work if git submodule is removed from the tree which leaves the submodule directory in workspace even after 'git clean -dxf'.

      If another '-f' is added, also the removed submodule will be deleted which is the behavior I'd expect when git submodules are in use.

      So Jenkins git plugin should call 'git clean -dxf -f' in that case.

      Pipelines workspace reuse

      An alternative use case, which I believe is very common, is that a pipeline checks out a repository in a sub-directory. Since the workspaces are reused between builds, there is a big risk that code or generated files in that sub-directory from previous builds are still there in the next run. This can seriously impact reliability of build or even fail them. One case is that old cucumber test reports are found.

      Completely wiping the workspace has a performance cost that is unreasonable.

          [JENKINS-26660] Add another '-f' to git clean to remove deleted submodules or other repos checked out in workspace in pipeline

          Mikko Rapeli created issue -

          Joshua Spence added a comment -

          +1

          Joshua Spence added a comment - +1
          Mark Waite made changes -
          Assignee Original: Nicolas De Loof [ ndeloof ]

          Tetyana Pylat added a comment -

          git clean -ffdx for "Clean after checkout" is exactly what we need. git clean -fdx doesn't handle case if any sobmodule was removed. And it cause a harm for our job because Untracked files are in repository

          Tetyana Pylat added a comment - git clean -ffdx for "Clean after checkout" is exactly what we need. git clean -fdx doesn't handle case if any sobmodule was removed. And it cause a harm for our job because Untracked files are in repository
          Tetyana Pylat made changes -
          Priority Original: Minor [ 4 ] New: Major [ 3 ]
          R. Tyler Croy made changes -
          Workflow Original: JNJira [ 160733 ] New: JNJira + In-Review [ 180468 ]
          Marcus Philip made changes -
          Component/s New: git-client-plugin [ 17423 ]
          Marcus Philip made changes -
          Component/s Original: git-plugin [ 15543 ]

          Marcus Philip added a comment -

          I believe this is in the git client plugin so I changed the component for this issue.

          Marcus Philip added a comment - I believe this is in the git client plugin so I changed the component for this issue.

          Marcus Philip added a comment - - edited

          In essence the fix is really simple and belongs to the epic category of bugs/features where the actual change is only one character Well, if there hadn't been 2 implementations...

          CLI

          Git cli clean help: "Git will refuse to delete directories with .git sub directory or file unless a second -f is given.

          An 'f' needs to be added here:
          CliGitAPIImpl.java Line: 671

          JGit

          org.eclipse.jgit.api.CleanCommand javadoc: 'If force is set, directories that are git repositories will also be deleted.'

          And '.setForce(true)' needs to be added here:
          JGitAPIImpl.java Line: 1300

          I claim this is a bug and not a feature request, because I think in the context of CI I think when you say 'Clean!' you really mean 'Clean everything!'. Thus we can change the behavior without providing any options. Can you imagine any sane use case relying on he current behavior?

          Marcus Philip added a comment - - edited In essence the fix is really simple and belongs to the epic category of bugs/features where the actual change is only one character Well, if there hadn't been 2 implementations... CLI Git cli clean help: "Git will refuse to delete directories with .git sub directory or file unless a second -f is given. An ' f ' needs to be added here: CliGitAPIImpl.java Line: 671 JGit org.eclipse.jgit.api.CleanCommand javadoc: 'If force is set, directories that are git repositories will also be deleted.' And ' .setForce(true) ' needs to be added here: JGitAPIImpl.java Line: 1300 I claim this is a bug and not a feature request, because I think in the context of CI I think when you say 'Clean!' you really mean 'Clean everything!'. Thus we can change the behavior without providing any options. Can you imagine any sane use case relying on he current behavior?

            Unassigned Unassigned
            mcfrisk Mikko Rapeli
            Votes:
            23 Vote for this issue
            Watchers:
            31 Start watching this issue

              Created:
              Updated:
              Resolved: