Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-26660

Add another '-f' to git clean to remove deleted submodules or other repos checked out in workspace in pipeline

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • None
    • Jenkins and multi SCM plugin configured with git repository using submodules
      Alternatively: A pipeline that checks out a second repository in a sub-dir
    • git-client-plugin 3.0.0, git-plugin 4.1.0

      Submodules clean up

      Jenkins SCM plugin can handle git repositories with submodules except for clean up.

      When "Clean before checkout" is selected, it runs 'git reset --hard && git clean -dxf' on the repository and submodules if "Recursively update submodules" is selected. This works for most stuff but it does not work if git submodule is removed from the tree which leaves the submodule directory in workspace even after 'git clean -dxf'.

      If another '-f' is added, also the removed submodule will be deleted which is the behavior I'd expect when git submodules are in use.

      So Jenkins git plugin should call 'git clean -dxf -f' in that case.

      Pipelines workspace reuse

      An alternative use case, which I believe is very common, is that a pipeline checks out a repository in a sub-directory. Since the workspaces are reused between builds, there is a big risk that code or generated files in that sub-directory from previous builds are still there in the next run. This can seriously impact reliability of build or even fail them. One case is that old cucumber test reports are found.

      Completely wiping the workspace has a performance cost that is unreasonable.

          [JENKINS-26660] Add another '-f' to git clean to remove deleted submodules or other repos checked out in workspace in pipeline

          Mikko Rapeli created issue -
          Mark Waite made changes -
          Assignee Original: Nicolas De Loof [ ndeloof ]
          Tetyana Pylat made changes -
          Priority Original: Minor [ 4 ] New: Major [ 3 ]
          R. Tyler Croy made changes -
          Workflow Original: JNJira [ 160733 ] New: JNJira + In-Review [ 180468 ]
          Marcus Philip made changes -
          Component/s New: git-client-plugin [ 17423 ]
          Marcus Philip made changes -
          Component/s Original: git-plugin [ 15543 ]
          Marcus Philip made changes -
          Summary Original: Add another '-f' to git clean to remove deleted submodules New: Add another '-f' to git clean to remove deleted submodules or other repos checked out in workspace in pipeline
          Marcus Philip made changes -
          Environment Original: Jenkins and multi SCM plugin configured with git repository using submodules New: Jenkins and multi SCM plugin configured with git repository using submodules
          Alternatively: A pipeline that checks out a second repository in a sub-dir
          Marcus Philip made changes -
          Description Original: Jenkins SCM plugin can handle git repositories with submodules except for clean up.

          When "Clean before checkout" is selected, it runs 'git reset --hard && git clean -dxf' on the repository and submodules if "Recursively update submodules" is selected. This works for most stuff but it does not work if git submodule is removed from the tree which leaves the submodule directory in workspace even after 'git clean -dxf'.

          If another '-f' is added, also the removed submodule will be deleted which is the behavior I'd expect when git submodules are in use.

          So Jenkins git plugin should call 'git clean -dxf -f' in that case.
          New: h4. Submodules clean up
          Jenkins SCM plugin can handle git repositories with submodules except for clean up.

          When "Clean before checkout" is selected, it runs 'git reset --hard && git clean -dxf' on the repository and submodules if "Recursively update submodules" is selected. This works for most stuff but it does not work if git submodule is removed from the tree which leaves the submodule directory in workspace even after 'git clean -dxf'.

          If another '-f' is added, also the removed submodule will be deleted which is the behavior I'd expect when git submodules are in use.

          So Jenkins git plugin should call 'git clean -dxf -f' in that case.

          h4. Pipelines workspace reuse
          An alternative use case, which I believe is very common, is that a pipeline checks out a repository in a sub-directory. Since the workspaces are reused between builds, there is a big risk that code or generated files in that sub-directory from previous builds are still there. This can seriously impact reliability of build or even fail them. One case is that old cucumber test reports are found.

          Completely wiping the workspace has a performance cost that is unreasonable.
          Marcus Philip made changes -
          Description Original: h4. Submodules clean up
          Jenkins SCM plugin can handle git repositories with submodules except for clean up.

          When "Clean before checkout" is selected, it runs 'git reset --hard && git clean -dxf' on the repository and submodules if "Recursively update submodules" is selected. This works for most stuff but it does not work if git submodule is removed from the tree which leaves the submodule directory in workspace even after 'git clean -dxf'.

          If another '-f' is added, also the removed submodule will be deleted which is the behavior I'd expect when git submodules are in use.

          So Jenkins git plugin should call 'git clean -dxf -f' in that case.

          h4. Pipelines workspace reuse
          An alternative use case, which I believe is very common, is that a pipeline checks out a repository in a sub-directory. Since the workspaces are reused between builds, there is a big risk that code or generated files in that sub-directory from previous builds are still there. This can seriously impact reliability of build or even fail them. One case is that old cucumber test reports are found.

          Completely wiping the workspace has a performance cost that is unreasonable.
          New: h4. Submodules clean up
          Jenkins SCM plugin can handle git repositories with submodules except for clean up.

          When "Clean before checkout" is selected, it runs 'git reset --hard && git clean -dxf' on the repository and submodules if "Recursively update submodules" is selected. This works for most stuff but it does not work if git submodule is removed from the tree which leaves the submodule directory in workspace even after 'git clean -dxf'.

          If another '-f' is added, also the removed submodule will be deleted which is the behavior I'd expect when git submodules are in use.

          So Jenkins git plugin should call 'git clean -dxf -f' in that case.

          h4. Pipelines workspace reuse
          An alternative use case, which I believe is very common, is that a pipeline checks out a repository in a sub-directory. Since the workspaces are reused between builds, there is a big risk that code or generated files in that sub-directory from previous builds are still there in the next run. This can seriously impact reliability of build or even fail them. One case is that old cucumber test reports are found.

          Completely wiping the workspace has a performance cost that is unreasonable.
          Mark Waite made changes -
          Link New: This issue is duplicated by JENKINS-45705 [ JENKINS-45705 ]

            Unassigned Unassigned
            mcfrisk Mikko Rapeli
            Votes:
            23 Vote for this issue
            Watchers:
            31 Start watching this issue

              Created:
              Updated:
              Resolved: