Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-13165

Cloning workspace loses hidden files/directories

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Duplicate
    • Component/s: core
    • Labels:
      None
    • Environment:
      Linux
    • Similar Issues:

      Description

      I upgraded to Jenkins 1.456 today, and now my jobs that use the Clone Workspace plugin are having problems because the workspace tar.gz that is created when the job completes no longer includes the .git directory. In 1.449, which I ended up downgrading back to, the default setting for files to include always pulls that in. I've verified this by running tar -tzf on the workspace.tar.gz; in 1.449 .git is there, in 1.456 it isn't.

      I wasn't able to force Jenkins to include it either. I tried putting "**/, **/.git" and other variations on that in the list of files to include, but none of them seemed to work.

        Attachments

          Issue Links

            Activity

            owenmehegan Owen Mehegan created issue -
            Hide
            kohsuke Kohsuke Kawaguchi added a comment -

            This is about dot files, not the hidden file flag in NTFS/FAT. Just to be sure.

            Show
            kohsuke Kohsuke Kawaguchi added a comment - This is about dot files, not the hidden file flag in NTFS/FAT. Just to be sure.
            owenmehegan Owen Mehegan made changes -
            Field Original Value New Value
            Assignee Kohsuke Kawaguchi [ kohsuke ]
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Kohsuke Kawaguchi
            Path:
            test/src/test/java/hudson/FileSystemProvisionerTest.java
            http://jenkins-ci.org/commit/jenkins/2b5a24c9b0ec6c3f1feedde080aa3196f419ae40
            Log:
            JENKINS-13165 Added a test to reproduce the problem to no avail.

            Compare: https://github.com/jenkinsci/jenkins/compare/196aadc...2b5a24c

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: test/src/test/java/hudson/FileSystemProvisionerTest.java http://jenkins-ci.org/commit/jenkins/2b5a24c9b0ec6c3f1feedde080aa3196f419ae40 Log: JENKINS-13165 Added a test to reproduce the problem to no avail. Compare: https://github.com/jenkinsci/jenkins/compare/196aadc...2b5a24c
            Hide
            kohsuke Kohsuke Kawaguchi added a comment -

            As my commit indicates, I fail to reproduce this problem. Hmm.

            Show
            kohsuke Kohsuke Kawaguchi added a comment - As my commit indicates, I fail to reproduce this problem. Hmm.
            Hide
            dogfood dogfood added a comment -

            Integrated in jenkins_main_trunk #1605
            JENKINS-13165 Added a test to reproduce the problem to no avail. (Revision 2b5a24c9b0ec6c3f1feedde080aa3196f419ae40)

            Result = SUCCESS
            Kohsuke Kawaguchi : 2b5a24c9b0ec6c3f1feedde080aa3196f419ae40
            Files :

            • test/src/test/java/hudson/FileSystemProvisionerTest.java
            Show
            dogfood dogfood added a comment - Integrated in jenkins_main_trunk #1605 JENKINS-13165 Added a test to reproduce the problem to no avail. (Revision 2b5a24c9b0ec6c3f1feedde080aa3196f419ae40) Result = SUCCESS Kohsuke Kawaguchi : 2b5a24c9b0ec6c3f1feedde080aa3196f419ae40 Files : test/src/test/java/hudson/FileSystemProvisionerTest.java
            Hide
            commana Christoph Thelen added a comment -

            I have the same problem. To reproduce it I downloaded 1.457, installed "Jenkins Git Plugin" and "Jenkins Clone Workspace SCM Plugin" and set up two Jobs as follows:

            • "GitJob" using a Git repository, then archiving the workspace and triggering the next job.
            • "CloneJob" using the previously archived workspace as SCM.

            CloneJob's workspace will not contain the Git files (.git directory as well as all .gitignore files). Other dot-files are unaffected (eg. Eclipse .project files will be present). As owenmehegan stated, Git files were present in previous Jenkins versions.

            Show
            commana Christoph Thelen added a comment - I have the same problem. To reproduce it I downloaded 1.457, installed "Jenkins Git Plugin" and "Jenkins Clone Workspace SCM Plugin" and set up two Jobs as follows: "GitJob" using a Git repository, then archiving the workspace and triggering the next job. "CloneJob" using the previously archived workspace as SCM. CloneJob's workspace will not contain the Git files (.git directory as well as all .gitignore files). Other dot-files are unaffected (eg. Eclipse .project files will be present). As owenmehegan stated, Git files were present in previous Jenkins versions.
            Hide
            commana Christoph Thelen added a comment -

            Some further details: The bug first appears in version 1.456; version 1.455 works fine. I even tested some older versions (1.449, 1.450, 1.454) which all work as expected. In all tests I used the same plugin versions (1.1.16 for Git and 0.4 for Clone Workspace).

            Show
            commana Christoph Thelen added a comment - Some further details: The bug first appears in version 1.456; version 1.455 works fine. I even tested some older versions (1.449, 1.450, 1.454) which all work as expected. In all tests I used the same plugin versions (1.1.16 for Git and 0.4 for Clone Workspace).
            Hide
            commana Christoph Thelen added a comment -

            For the records: Since Ant 1.8.2, Git files/directories (and those of other SCMs) are included in the "default excludes", which is why the files are lost in a cloned workspace. Therefore this is not a Jenkins "bug" but an Ant "feature". I have opened a pull request allowing the disabling of those default excludes. Plugins like Clone Workspace may use this to let Git files reappear.

            Show
            commana Christoph Thelen added a comment - For the records: Since Ant 1.8.2, Git files/directories (and those of other SCMs) are included in the "default excludes", which is why the files are lost in a cloned workspace. Therefore this is not a Jenkins "bug" but an Ant "feature". I have opened a pull request allowing the disabling of those default excludes. Plugins like Clone Workspace may use this to let Git files reappear.
            Hide
            dogfood dogfood added a comment -

            Integrated in jenkins_ui-changes_branch #21
            JENKINS-13165 Added a test to reproduce the problem to no avail. (Revision 2b5a24c9b0ec6c3f1feedde080aa3196f419ae40)

            Result = SUCCESS
            Kohsuke Kawaguchi : 2b5a24c9b0ec6c3f1feedde080aa3196f419ae40
            Files :

            • test/src/test/java/hudson/FileSystemProvisionerTest.java
            Show
            dogfood dogfood added a comment - Integrated in jenkins_ui-changes_branch #21 JENKINS-13165 Added a test to reproduce the problem to no avail. (Revision 2b5a24c9b0ec6c3f1feedde080aa3196f419ae40) Result = SUCCESS Kohsuke Kawaguchi : 2b5a24c9b0ec6c3f1feedde080aa3196f419ae40 Files : test/src/test/java/hudson/FileSystemProvisionerTest.java
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Christoph Thelen
            Path:
            changelog.html
            core/src/main/java/hudson/util/DirScanner.java
            core/src/test/java/hudson/util/DirScannerTest.java
            http://jenkins-ci.org/commit/jenkins/0725d2765da789e02914deb4893a449eeda6a820
            Log:
            [FIXED JENKINS-13165] Allow plugins to disable usage of Ant default excludes.

            This is also related to JENKINS-9778.

            Compare: https://github.com/jenkinsci/jenkins/compare/3c349c0...0725d27

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Thelen Path: changelog.html core/src/main/java/hudson/util/DirScanner.java core/src/test/java/hudson/util/DirScannerTest.java http://jenkins-ci.org/commit/jenkins/0725d2765da789e02914deb4893a449eeda6a820 Log: [FIXED JENKINS-13165] Allow plugins to disable usage of Ant default excludes. This is also related to JENKINS-9778 . Compare: https://github.com/jenkinsci/jenkins/compare/3c349c0...0725d27
            Hide
            dogfood dogfood added a comment -

            Integrated in jenkins_main_trunk #1681

            Result = SUCCESS

            Show
            dogfood dogfood added a comment - Integrated in jenkins_main_trunk #1681 Result = SUCCESS
            Hide
            dogfood dogfood added a comment -

            Integrated in jenkins_ui-changes_branch #26
            [FIXED JENKINS-13165] Allow plugins to disable usage of Ant default excludes. (Revision 0725d2765da789e02914deb4893a449eeda6a820)

            Result = SUCCESS
            Kohsuke Kawaguchi : 0725d2765da789e02914deb4893a449eeda6a820
            Files :

            • core/src/test/java/hudson/util/DirScannerTest.java
            • core/src/main/java/hudson/util/DirScanner.java
            • changelog.html
            Show
            dogfood dogfood added a comment - Integrated in jenkins_ui-changes_branch #26 [FIXED JENKINS-13165] Allow plugins to disable usage of Ant default excludes. (Revision 0725d2765da789e02914deb4893a449eeda6a820) Result = SUCCESS Kohsuke Kawaguchi : 0725d2765da789e02914deb4893a449eeda6a820 Files : core/src/test/java/hudson/util/DirScannerTest.java core/src/main/java/hudson/util/DirScanner.java changelog.html
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Olivier Lamy
            Path:
            changelog.html
            http://jenkins-ci.org/commit/jenkins/d7d9cf4a8618e777c2303aec010f5443f13aeaf6
            Log:
            changelog entry for JENKINS-13165

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Olivier Lamy Path: changelog.html http://jenkins-ci.org/commit/jenkins/d7d9cf4a8618e777c2303aec010f5443f13aeaf6 Log: changelog entry for JENKINS-13165
            Hide
            whimboo Henrik Skupin added a comment -

            A pull request is up on github for 2 months now. Could anyone with privileges please check it and get landed if appropriate? Our work is massively blocked on a new version of this plugin.

            https://github.com/jenkinsci/jenkins-clone-workspace-scm-plugin/commit/0bfecdad2fddc44c43b96f88ed373ab225e330e6

            Thanks a lot!

            Show
            whimboo Henrik Skupin added a comment - A pull request is up on github for 2 months now. Could anyone with privileges please check it and get landed if appropriate? Our work is massively blocked on a new version of this plugin. https://github.com/jenkinsci/jenkins-clone-workspace-scm-plugin/commit/0bfecdad2fddc44c43b96f88ed373ab225e330e6 Thanks a lot!
            Hide
            hashar Antoine Musso added a comment -

            I have opened JENKINS-14355 about the fingerprint plugin not being able to fingerprint ".git/FETCH_HEAD. I believe it is a similar issue as dirscanner fixed by https://github.com/jenkinsci/jenkins/commit/0725d2765da789e02914deb4893a449eeda6a820#L1R102 but with Util.createFileSet() this time.

            Show
            hashar Antoine Musso added a comment - I have opened JENKINS-14355 about the fingerprint plugin not being able to fingerprint ".git/FETCH_HEAD. I believe it is a similar issue as dirscanner fixed by https://github.com/jenkinsci/jenkins/commit/0725d2765da789e02914deb4893a449eeda6a820#L1R102 but with Util.createFileSet() this time.
            jglick Jesse Glick made changes -
            Link This issue is related to JENKINS-14355 [ JENKINS-14355 ]
            jglick Jesse Glick made changes -
            Link This issue is related to JENKINS-13888 [ JENKINS-13888 ]
            jglick Jesse Glick made changes -
            Link This issue is related to JENKINS-14900 [ JENKINS-14900 ]
            Hide
            davehunt Dave Hunt added a comment -

            Please correct me if I'm wrong, but I believe this was fixed by JENKINS-13888

            Show
            davehunt Dave Hunt added a comment - Please correct me if I'm wrong, but I believe this was fixed by JENKINS-13888
            davehunt Dave Hunt made changes -
            Resolution Duplicate [ 3 ]
            Status Open [ 1 ] Resolved [ 5 ]
            Hide
            whimboo Henrik Skupin added a comment -

            I tested this with the last Jenkins LTS version (1.509.2) and the Jenkins Clone Workspace SCM Plug-in version 0.5. Everything works fine now and the hidden files are included in the workspace archive. So this is indeed fixed!

            Thanks!

            Show
            whimboo Henrik Skupin added a comment - I tested this with the last Jenkins LTS version (1.509.2) and the Jenkins Clone Workspace SCM Plug-in version 0.5. Everything works fine now and the hidden files are included in the workspace archive. So this is indeed fixed! Thanks!
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 143629 ] JNJira + In-Review [ 190649 ]
            saper Marcin CieĊ›lak made changes -
            Link This issue is related to JENKINS-7999 [ JENKINS-7999 ]

              People

              Assignee:
              kohsuke Kohsuke Kawaguchi
              Reporter:
              owenmehegan Owen Mehegan
              Votes:
              5 Vote for this issue
              Watchers:
              9 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: