Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-10629

Tar implimentation can't handle > 8GB and doesn't error out.

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Minor Minor
    • core

      During archiving from a slave the untaring can fail with files > 8GB. This is because the tar creator makes a corrupt archive. I think it doesn't properly null terminate the size value when the file size is gets to the 12th digit - which is a violation of tar spec (from Wikipedia tar article: Numeric values are encoded in octal numbers using ASCII digits, with leading zeroes. For historical reasons, a final NUL or space character should be used. Thus although there are 12 bytes reserved for storing the file size, only 11 octal digits can be stored. This gives a maximum file size of 8 gigabytes on archived files.)
      Changing to gnutar might be nice to handle large files, or at least making the library error out when it hits large files rather than happily building corrupt archives.

      Error caused by this bug (due to reading the corrupt header):

      Archiving artifacts
      ERROR: Publisher hudson.tasks.ArtifactArchiver aborted due to exception
      java.lang.IllegalArgumentException: Negative time
      at java.io.File.setLastModified(File.java:1258)
      at hudson.FilePath.readFromTar(FilePath.java:1605)
      at hudson.FilePath.copyRecursiveTo(FilePath.java:1530)
      at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:117)
      at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
      at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:649)
      at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:625)
      at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:603)
      at hudson.model.Build$RunnerImpl.post2(Build.java:161)
      at hudson.model.AbstractBuild$AbstractRunner.post(AbstractBuild.java:572)
      at hudson.model.Run.run(Run.java:1386)
      at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
      at hudson.model.ResourceController.execute(ResourceController.java:88)
      at hudson.model.Executor.run(Executor.java:145)
      Finished: FAILURE

          [JENKINS-10629] Tar implimentation can't handle > 8GB and doesn't error out.

          keith baker created issue -

          Alan Harder added a comment -

          tar functionality is in core, not within copyartifact plugin.. reassigning component.

          Alan Harder added a comment - tar functionality is in core, not within copyartifact plugin.. reassigning component.
          Alan Harder made changes -
          Component/s New: core [ 15593 ]
          Component/s Original: copyartifact [ 15692 ]
          Assignee Original: Alan Harder [ mindless ]

          any solution and workaorund for this issue.
          still we are facing this issue while copying workspace of another project by "Copy artifacts from another project"

          hiteswar kumar added a comment - any solution and workaorund for this issue. still we are facing this issue while copying workspace of another project by "Copy artifacts from another project"

          I am experiencing this problem as well. How does jenkins invoke tar on a windows slave?

          don fitzgerald added a comment - I am experiencing this problem as well. How does jenkins invoke tar on a windows slave?

          Daniel Beck added a comment -

          Jenkins uses a patched variant of Ant's org.apache.tools.tar.TarInputStream imported from Ant 1.7.0:
          https://github.com/jenkinsci/jenkins/tree/master/core/src/main/java/hudson/org/apache/tools/tar

          Maybe it's time to update this a bit, there've been a few changes since – cannot tell whether relevant to this issue yet though:
          http://svn.apache.org/viewvc/ant/core/trunk/src/main/org/apache/tools/tar/TarInputStream.java?view=log

          Daniel Beck added a comment - Jenkins uses a patched variant of Ant's org.apache.tools.tar.TarInputStream imported from Ant 1.7.0: https://github.com/jenkinsci/jenkins/tree/master/core/src/main/java/hudson/org/apache/tools/tar Maybe it's time to update this a bit, there've been a few changes since – cannot tell whether relevant to this issue yet though: http://svn.apache.org/viewvc/ant/core/trunk/src/main/org/apache/tools/tar/TarInputStream.java?view=log

          This is critical for my project. Is there any workaround?

          Akihiro KAYAMA added a comment - This is critical for my project. Is there any workaround?

          Daniel Beck added a comment -

          kayama: Create smaller files (e.g. splitting them up for archiving every 7GB, or packaging them using a tool that can split archives.

          Daniel Beck added a comment - kayama : Create smaller files (e.g. splitting them up for archiving every 7GB, or packaging them using a tool that can split archives.

          Thanks for your comment. I have a single large proprietary package file as artifact. Splitting it is inconvenience but I understand there is no other way.

          Akihiro KAYAMA added a comment - Thanks for your comment. I have a single large proprietary package file as artifact. Splitting it is inconvenience but I understand there is no other way.
          Daniel Beck made changes -
          Link New: This issue is duplicated by JENKINS-25006 [ JENKINS-25006 ]

            integer Kanstantsin Shautsou
            keibaker keith baker
            Votes:
            6 Vote for this issue
            Watchers:
            26 Start watching this issue

              Created:
              Updated:
              Resolved: