• Icon: New Feature New Feature
    • Resolution: Fixed
    • Icon: Major Major
    • core
    • None

      Currently artifact archiving follows a fixed model: artifacts are defined in ArtifactArchiver (or MavenArtifactArchiver) configuration; they are copied from slave to master using FilePath.copyRecursiveTo; they are stored in ${Run.rootDir}/archive; and they are served by browsing this directory in master.

      This system has obvious scalability limitations: channel transfer is not particularly efficient; and storing enormous files in the builds directory can cause disk overruns.

      The CloudBees Fast Archiver plugin tries to address the transfer speed issue by using a better algorithm. This helps, but the implementation (replacing the ArtifactArchiver build step) is very awkward, and cannot be made to work properly with Maven projects, fingerprints, etc.

      Some installations may anyway not need to copy files at all. The master and slave might share a common network filesystem, or might have a way of efficiently transferring file references (e.g. ZFS send).

      Finally, there should be the option to store artifacts in a better way: on a big slow drive, in a compressed archive, etc.

      To handle all these requirements we need an extension point that encompasses both transfer and storage of artifacts for each build.

          [JENKINS-17236] Pluggable artifact transfer & storage

          Jesse Glick created issue -
          Jesse Glick made changes -
          Link New: This issue is related to JENKINS-7813 [ JENKINS-7813 ]
          Jesse Glick made changes -
          Link New: This issue is related to JENKINS-10502 [ JENKINS-10502 ]

          Code changed in jenkins
          User: Jesse Glick
          Path:
          core/src/main/java/hudson/model/Run.java
          core/src/main/java/hudson/tasks/ArtifactArchiver.java
          core/src/main/java/jenkins/model/ArtifactManager.java
          core/src/main/java/jenkins/model/StandardArtifactManager.java
          maven-plugin/src/main/java/hudson/maven/MavenBuild.java
          maven-plugin/src/main/java/hudson/maven/MavenBuildProxy.java
          maven-plugin/src/main/java/hudson/maven/reporters/MavenArtifact.java
          maven-plugin/src/main/java/hudson/maven/reporters/MavenArtifactArchiver.java
          maven-plugin/src/main/java/hudson/maven/reporters/MavenArtifactRecord.java
          http://jenkins-ci.org/commit/jenkins/e1eea67bb2fa1a356492e91313d84a523f441b34
          Log:
          JENKINS-17236 Initial version of ArtifactManager (no Maven support yet).


          You received this message because you are subscribed to the Google Groups "Jenkins Commits" group.
          To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-commits+unsubscribe@googlegroups.com.
          For more options, visit https://groups.google.com/groups/opt_out.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/hudson/model/Run.java core/src/main/java/hudson/tasks/ArtifactArchiver.java core/src/main/java/jenkins/model/ArtifactManager.java core/src/main/java/jenkins/model/StandardArtifactManager.java maven-plugin/src/main/java/hudson/maven/MavenBuild.java maven-plugin/src/main/java/hudson/maven/MavenBuildProxy.java maven-plugin/src/main/java/hudson/maven/reporters/MavenArtifact.java maven-plugin/src/main/java/hudson/maven/reporters/MavenArtifactArchiver.java maven-plugin/src/main/java/hudson/maven/reporters/MavenArtifactRecord.java http://jenkins-ci.org/commit/jenkins/e1eea67bb2fa1a356492e91313d84a523f441b34 Log: JENKINS-17236 Initial version of ArtifactManager (no Maven support yet). – You received this message because you are subscribed to the Google Groups "Jenkins Commits" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-commits+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out .
          Jesse Glick made changes -
          Jesse Glick made changes -
          Status Original: Open [ 1 ] New: In Progress [ 3 ]

          Jesse Glick added a comment -

          API should be fleshed out; still need to add Maven support. Working on a prototype plugin archiving to ${build.rootDir}/archive.zip.

          Jesse Glick added a comment - API should be fleshed out; still need to add Maven support. Working on a prototype plugin archiving to ${build.rootDir}/archive.zip .
          Jesse Glick made changes -
          Link New: This issue is blocking JENKINS-6229 [ JENKINS-6229 ]

          Code changed in jenkins
          User: Kohsuke Kawaguchi
          Path:
          changelog.html
          core/src/main/java/hudson/FilePath.java
          core/src/main/java/hudson/model/DirectoryBrowserSupport.java
          core/src/main/java/hudson/model/Run.java
          core/src/main/java/hudson/tasks/ArtifactArchiver.java
          core/src/main/java/hudson/util/DirScanner.java
          core/src/main/java/jenkins/model/ArtifactManager.java
          core/src/main/java/jenkins/model/ArtifactManagerConfiguration.java
          core/src/main/java/jenkins/model/ArtifactManagerFactory.java
          core/src/main/java/jenkins/model/ArtifactManagerFactoryDescriptor.java
          core/src/main/java/jenkins/model/StandardArtifactManager.java
          core/src/main/java/jenkins/util/VirtualFile.java
          core/src/main/resources/jenkins/model/ArtifactManagerConfiguration/config.groovy
          core/src/test/java/hudson/model/RunTest.java
          maven-plugin/src/main/java/hudson/maven/MavenBuild.java
          maven-plugin/src/main/java/hudson/maven/MavenBuildProxy.java
          maven-plugin/src/main/java/hudson/maven/MavenModuleSetBuild.java
          maven-plugin/src/main/java/hudson/maven/reporters/MavenArtifact.java
          maven-plugin/src/main/java/hudson/maven/reporters/MavenArtifactArchiver.java
          maven-plugin/src/main/java/hudson/maven/reporters/MavenArtifactRecord.java
          maven-plugin/src/test/java/hudson/maven/reporters/SurefireArchiverUnitTest.java
          test/src/test/java/hudson/maven/MavenMultiModuleTest.java
          test/src/test/java/hudson/model/RunTest.java
          http://jenkins-ci.org/commit/jenkins/b698b672ec5d4033115d84b2bc2b179812080bdd
          Log:
          [FIXED JENKINS-17236] Introduce ArtifactManager

          Merge branch 'ArtifactManager-JENKINS-17236'

          Compare: https://github.com/jenkinsci/jenkins/compare/eea972ec11a1...b698b672ec5d

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: changelog.html core/src/main/java/hudson/FilePath.java core/src/main/java/hudson/model/DirectoryBrowserSupport.java core/src/main/java/hudson/model/Run.java core/src/main/java/hudson/tasks/ArtifactArchiver.java core/src/main/java/hudson/util/DirScanner.java core/src/main/java/jenkins/model/ArtifactManager.java core/src/main/java/jenkins/model/ArtifactManagerConfiguration.java core/src/main/java/jenkins/model/ArtifactManagerFactory.java core/src/main/java/jenkins/model/ArtifactManagerFactoryDescriptor.java core/src/main/java/jenkins/model/StandardArtifactManager.java core/src/main/java/jenkins/util/VirtualFile.java core/src/main/resources/jenkins/model/ArtifactManagerConfiguration/config.groovy core/src/test/java/hudson/model/RunTest.java maven-plugin/src/main/java/hudson/maven/MavenBuild.java maven-plugin/src/main/java/hudson/maven/MavenBuildProxy.java maven-plugin/src/main/java/hudson/maven/MavenModuleSetBuild.java maven-plugin/src/main/java/hudson/maven/reporters/MavenArtifact.java maven-plugin/src/main/java/hudson/maven/reporters/MavenArtifactArchiver.java maven-plugin/src/main/java/hudson/maven/reporters/MavenArtifactRecord.java maven-plugin/src/test/java/hudson/maven/reporters/SurefireArchiverUnitTest.java test/src/test/java/hudson/maven/MavenMultiModuleTest.java test/src/test/java/hudson/model/RunTest.java http://jenkins-ci.org/commit/jenkins/b698b672ec5d4033115d84b2bc2b179812080bdd Log: [FIXED JENKINS-17236] Introduce ArtifactManager Merge branch 'ArtifactManager- JENKINS-17236 ' Compare: https://github.com/jenkinsci/jenkins/compare/eea972ec11a1...b698b672ec5d
          Jesse Glick made changes -
          Resolution New: Fixed [ 1 ]
          Status Original: In Progress [ 3 ] New: Resolved [ 5 ]

            jglick Jesse Glick
            jglick Jesse Glick
            Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: