Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-20086

Allow disabling use of default exclude patterns (.git, .svn, etc.) for archiving

    • Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Major Major
    • core
    • Any

      The plugin available to archive artifacts doesn't allow to chose to archive hidden files/directories like .git, .svn, etc. This feature will help Continuous Integration teams to upstream the .git information to be used for example to get last commit hash, repository server, etc.
      Today, the plugin that copy the workspace is not useful when you do not need more than that artifacts, ans, in some cases, the repository is big.
      Maybe, the plugin can be changed to allow the copy, by selecting the file types.
      I am talking about to change the plugin "Archive Artifact" and "Copy Artifact".

          [JENKINS-20086] Allow disabling use of default exclude patterns (.git, .svn, etc.) for archiving

          mbaker000 added a comment -

          While it's a fair amount of hackery you can do this to work around this limitation:

          1. use 7-Zip to package up your workspace
          2. archive the zip file as an artifact
          3. copy the zip file as an artifact to your next pipeline build
          4. unzip the zip file
          5. do your normal build steps

          mbaker000 added a comment - While it's a fair amount of hackery you can do this to work around this limitation: 1. use 7-Zip to package up your workspace 2. archive the zip file as an artifact 3. copy the zip file as an artifact to your next pipeline build 4. unzip the zip file 5. do your normal build steps

          mike omoto added a comment -

          You can also emulate this behavior by using the post-build task plugin with the following setup:

          Log Text: Archiving artifacts

          cp -r .git ../builds/$BUILD_NUMBER/archive/.git/
          cp .gitignore ../builds/$BUILD_NUMBER/archive/.gitignore
          cp .gitattributes ../builds/$BUILD_NUMBER/archive/.gitattributes

          mike omoto added a comment - You can also emulate this behavior by using the post-build task plugin with the following setup: Log Text: Archiving artifacts cp -r .git ../builds/$BUILD_NUMBER/archive/.git/ cp .gitignore ../builds/$BUILD_NUMBER/archive/.gitignore cp .gitattributes ../builds/$BUILD_NUMBER/archive/.gitattributes

          Gigi Lentini added a comment -

          I came across the same issue using Jenkins 1.554.1, and the zip/unzip workaround seems to work for the moment.
          But I would rather use a clear jenkins config to do so.

          Do you have any update?

          Thanks.

          Gigi Lentini added a comment - I came across the same issue using Jenkins 1.554.1, and the zip/unzip workaround seems to work for the moment. But I would rather use a clear jenkins config to do so. Do you have any update? Thanks.

          Daniel Beck added a comment -

          Jenkins uses Ant to archive files. It excludes the following patterns related to various SCM by default (run in Script Console):

          org.apache.tools.ant.DirectoryScanner.defaultExcludes.each { println it }

          This can be changed globally by running the script below in the Script Console (until the next restart) or by adding a .groovy file with that script in JENKINS_HOME/init.groovy.d/ so this gets loaded upon startup:

          org.apache.tools.ant.DirectoryScanner.defaultExcludes.each { org.apache.tools.ant.DirectoryScanner.removeDefaultExclude(it) }

          I doubt that this is deserving a per-project preference, given the relative obscurity of the requirement. IMO a cleaner solution would be to extract the relevant data during the build and store it outside the SCM metadata folders.

          Daniel Beck added a comment - Jenkins uses Ant to archive files. It excludes the following patterns related to various SCM by default (run in Script Console): org.apache.tools.ant.DirectoryScanner.defaultExcludes.each { println it } This can be changed globally by running the script below in the Script Console (until the next restart) or by adding a .groovy file with that script in JENKINS_HOME/init.groovy.d/ so this gets loaded upon startup: org.apache.tools.ant.DirectoryScanner.defaultExcludes.each { org.apache.tools.ant.DirectoryScanner.removeDefaultExclude(it) } I doubt that this is deserving a per-project preference, given the relative obscurity of the requirement. IMO a cleaner solution would be to extract the relevant data during the build and store it outside the SCM metadata folders.

          relative obscurity of the requirement

          why is it obscurs?

          I have the same issue. First step clean/install the maven project on a private maven repo (to avoid snapshot deps collision between a reentrant pipeline) and archive the workspace. Next steps (test, sonar, deploy on nexus, deploy on tomcat...) reuse this archive instead of rebuilding the project. Sonar steps need the .svn which are not copied by the archive plugin.

          Is this pipeline wrong?

          Nicolas Labrot added a comment - relative obscurity of the requirement why is it obscurs? I have the same issue. First step clean/install the maven project on a private maven repo (to avoid snapshot deps collision between a reentrant pipeline) and archive the workspace. Next steps (test, sonar, deploy on nexus, deploy on tomcat...) reuse this archive instead of rebuilding the project. Sonar steps need the .svn which are not copied by the archive plugin. Is this pipeline wrong?

          Daniel Beck added a comment -

          It's not something that can be done out of the box, you need e.g. Copy Artifacts plugin for that.

          Still, it does seem like a somewhat surprising behavior – it's only unsurprising if you know it uses Ant, and that's a build tool. I'm not sure what the best way to introduce an option is though – just clearing the default excludes can well be considered a regression. Per-instance option is too heavy-handed. Per-job option seems to be the only sensible one left. Given the apparent reluctance of others to merge my 'archive only if successful' option in 1.567, I'm not looking forward to an argument about this

          I'm adding this to my list of things I'd like to fix, but anyone feel free to take over.

          Daniel Beck added a comment - It's not something that can be done out of the box, you need e.g. Copy Artifacts plugin for that. Still, it does seem like a somewhat surprising behavior – it's only unsurprising if you know it uses Ant, and that's a build tool. I'm not sure what the best way to introduce an option is though – just clearing the default excludes can well be considered a regression. Per-instance option is too heavy-handed. Per-job option seems to be the only sensible one left. Given the apparent reluctance of others to merge my 'archive only if successful' option in 1.567, I'm not looking forward to an argument about this I'm adding this to my list of things I'd like to fix, but anyone feel free to take over.

          As a workaround I have remove ".svn" from the defaultExcludes with an init.groovy hook.

          Per job option seems to me the right way. And maybe per copy/archive task (a checkbox to include defaultExcludes).

          Nicolas Labrot added a comment - As a workaround I have remove ".svn" from the defaultExcludes with an init.groovy hook. Per job option seems to me the right way. And maybe per copy/archive task (a checkbox to include defaultExcludes).

          Daniel Beck added a comment -

          Right, I meant per post-build step. As there's usually only one, it's effectively "per job".

          I don't know whether the same behavior applies to Copy Artifact plugin. I'd consider that a separate issue though, as it's a different component.

          Daniel Beck added a comment - Right, I meant per post-build step. As there's usually only one, it's effectively "per job". I don't know whether the same behavior applies to Copy Artifact plugin. I'd consider that a separate issue though, as it's a different component.

          Nicolas Labrot added a comment - - edited

          I'm giving it a try to pull request a modification of the ArtifactArchiver. I will check the copy artifact plugin.

          Nicolas Labrot added a comment - - edited I'm giving it a try to pull request a modification of the ArtifactArchiver . I will check the copy artifact plugin.

          Artifact archiver issue is fixed on github. I agree with you Daniel, this issue should be split

          Nicolas Labrot added a comment - Artifact archiver issue is fixed on github. I agree with you Daniel, this issue should be split

          Daniel Beck added a comment -

          Nicolas' pull request: https://github.com/jenkinsci/jenkins/pull/1287

          Unless someone else merges it, I'm going to let it soak for a few more days to allow others to provide feedback. This week is JUC Boston, next week JUC Berlin, so most reviewers will be distracted right now.

          Daniel Beck added a comment - Nicolas' pull request: https://github.com/jenkinsci/jenkins/pull/1287 Unless someone else merges it, I'm going to let it soak for a few more days to allow others to provide feedback. This week is JUC Boston, next week JUC Berlin, so most reviewers will be distracted right now.

          Code changed in jenkins
          User: nlabrot
          Path:
          core/src/main/java/hudson/tasks/ArtifactArchiver.java
          core/src/main/resources/hudson/tasks/ArtifactArchiver/config.jelly
          core/src/main/resources/hudson/tasks/ArtifactArchiver/config.properties
          core/src/main/resources/hudson/tasks/ArtifactArchiver/config_fr.properties
          core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes.groovy
          core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes.properties
          core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes_fr.properties
          test/src/test/java/hudson/tasks/ArtifactArchiverTest.java
          http://jenkins-ci.org/commit/jenkins/c726667162084e0199cea858735d35477d04feb9
          Log:
          [FIXED JENKINS-20086] Allow disabling use of default exclude patterns (.git, .svn, etc.)

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: nlabrot Path: core/src/main/java/hudson/tasks/ArtifactArchiver.java core/src/main/resources/hudson/tasks/ArtifactArchiver/config.jelly core/src/main/resources/hudson/tasks/ArtifactArchiver/config.properties core/src/main/resources/hudson/tasks/ArtifactArchiver/config_fr.properties core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes.groovy core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes.properties core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes_fr.properties test/src/test/java/hudson/tasks/ArtifactArchiverTest.java http://jenkins-ci.org/commit/jenkins/c726667162084e0199cea858735d35477d04feb9 Log: [FIXED JENKINS-20086] Allow disabling use of default exclude patterns (.git, .svn, etc.)

          Code changed in jenkins
          User: Oleg Nenashev
          Path:
          core/src/main/java/hudson/tasks/ArtifactArchiver.java
          core/src/main/resources/hudson/tasks/ArtifactArchiver/config.jelly
          core/src/main/resources/hudson/tasks/ArtifactArchiver/config.properties
          core/src/main/resources/hudson/tasks/ArtifactArchiver/config_fr.properties
          core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes.groovy
          core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes.properties
          core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes_fr.properties
          test/src/test/java/hudson/tasks/ArtifactArchiverTest.java
          http://jenkins-ci.org/commit/jenkins/6d5c25ae7e88fa2e870209b4fe2e55b7b75fc0aa
          Log:
          Merge pull request #1287 from nithril/master

          [FIXED JENKINS-20086] Allow disabling use of default exclude patterns (.git, .svn, etc.)

          Compare: https://github.com/jenkinsci/jenkins/compare/65bc0fb146bf...6d5c25ae7e88

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oleg Nenashev Path: core/src/main/java/hudson/tasks/ArtifactArchiver.java core/src/main/resources/hudson/tasks/ArtifactArchiver/config.jelly core/src/main/resources/hudson/tasks/ArtifactArchiver/config.properties core/src/main/resources/hudson/tasks/ArtifactArchiver/config_fr.properties core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes.groovy core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes.properties core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes_fr.properties test/src/test/java/hudson/tasks/ArtifactArchiverTest.java http://jenkins-ci.org/commit/jenkins/6d5c25ae7e88fa2e870209b4fe2e55b7b75fc0aa Log: Merge pull request #1287 from nithril/master [FIXED JENKINS-20086] Allow disabling use of default exclude patterns (.git, .svn, etc.) Compare: https://github.com/jenkinsci/jenkins/compare/65bc0fb146bf...6d5c25ae7e88

          dogfood added a comment -

          Integrated in jenkins_main_trunk #3490
          [FIXED JENKINS-20086] Allow disabling use of default exclude patterns (.git, .svn, etc.) (Revision c726667162084e0199cea858735d35477d04feb9)

          Result = SUCCESS
          nithril : c726667162084e0199cea858735d35477d04feb9
          Files :

          • core/src/main/resources/hudson/tasks/ArtifactArchiver/config_fr.properties
          • core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes.groovy
          • core/src/main/resources/hudson/tasks/ArtifactArchiver/config.properties
          • core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes_fr.properties
          • test/src/test/java/hudson/tasks/ArtifactArchiverTest.java
          • core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes.properties
          • core/src/main/java/hudson/tasks/ArtifactArchiver.java
          • core/src/main/resources/hudson/tasks/ArtifactArchiver/config.jelly

          dogfood added a comment - Integrated in jenkins_main_trunk #3490 [FIXED JENKINS-20086] Allow disabling use of default exclude patterns (.git, .svn, etc.) (Revision c726667162084e0199cea858735d35477d04feb9) Result = SUCCESS nithril : c726667162084e0199cea858735d35477d04feb9 Files : core/src/main/resources/hudson/tasks/ArtifactArchiver/config_fr.properties core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes.groovy core/src/main/resources/hudson/tasks/ArtifactArchiver/config.properties core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes_fr.properties test/src/test/java/hudson/tasks/ArtifactArchiverTest.java core/src/main/resources/hudson/tasks/ArtifactArchiver/help-defaultExcludes.properties core/src/main/java/hudson/tasks/ArtifactArchiver.java core/src/main/resources/hudson/tasks/ArtifactArchiver/config.jelly

          Gordon Jess added a comment -

          Sorry, still experiencing the issue, 

          pipeline code:

          archiveArtifacts artifacts: '*/', defaultExcludes: false, onlyIfSuccessful: true

          Gordon Jess added a comment - Sorry, still experiencing the issue,  pipeline code: archiveArtifacts artifacts: '*/', defaultExcludes: false, onlyIfSuccessful: true

          Daniel Beck added a comment -

          jess Works for me on Jenkins 2.89.3 and up to date plugins (sample Jenkinsfile: https://github.com/daniel-beck/pipeline-test/blob/43a0602597dd80995afa1714928a4b0e643ff18e/Jenkinsfile ). Please make sure you're not trying to archive in the wrong directory. Please file a new bug with complete and detailed steps to reproduce.

          Daniel Beck added a comment - jess Works for me on Jenkins 2.89.3 and up to date plugins (sample Jenkinsfile: https://github.com/daniel-beck/pipeline-test/blob/43a0602597dd80995afa1714928a4b0e643ff18e/Jenkinsfile ). Please make sure you're not trying to archive in the wrong directory. Please file a new bug with complete and detailed steps to reproduce.

            nithril Nicolas Labrot
            eduardodias Eduardo Dias
            Votes:
            1 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: