Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-23507

FilePath.installIfNecessaryFrom does not follow HTTP 30x redirects

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • core
    • None

      I'm using a tool installer which, for some downloads, the URL returns a 302 redirect.

      However, Jenkins reads the contents of the redirect page (i.e. some text like "This page has moved to http://foo/bar") and attempts to parse that as the zip/tar/whatever file.

      The stacktrace users see when this happens is pretty cryptic:

      Unpacking Go from http://golang.org/dl/go1.3.darwin-amd64-osx10.6.tar.gz to /Users/Shared/Jenkins/Home/tools/org.jenkinsci.plugins.golang.GolangInstallation/Go_1.3 on Jenkins
      FATAL: Failed to install http://golang.org/dl/go1.3.darwin-amd64-osx10.6.tar.gz to /Users/Shared/Jenkins/Home/tools/org.jenkinsci.plugins.golang.GolangInstallation/Go_1.3
      hudson.util.IOException2: Failed to install http://golang.org/dl/go1.3.darwin-amd64-osx10.6.tar.gz to /Users/Shared/Jenkins/Home/tools/org.jenkinsci.plugins.golang.GolangInstallation/Go_1.3
      	at hudson.FilePath.installIfNecessaryFrom(FilePath.java:768)
      	at org.jenkinsci.plugins.golang.GolangInstaller.performInstallation(GolangInstaller.java:53)
      	at hudson.tools.InstallerTranslator.getToolHome(InstallerTranslator.java:61)
      	at hudson.tools.ToolLocationNodeProperty.getToolHome(ToolLocationNodeProperty.java:107)
      	at hudson.tools.ToolInstallation.translateFor(ToolInstallation.java:204)
      	at org.jenkinsci.plugins.golang.GolangInstallation.forNode(GolangInstallation.java:41)
      	at org.jenkinsci.plugins.golang.GolangBuildWrapper.setUp(GolangBuildWrapper.java:34)
              ...
      Caused by: hudson.util.IOException2: Failed to unpack http://golang.org/dl/go1.3.darwin-amd64-osx10.6.tar.gz (28 bytes read of total -1)
      	at hudson.FilePath.installIfNecessaryFrom(FilePath.java:762)
      	... 12 more
      Caused by: hudson.util.IOException2: Failed to extract input stream
      	at hudson.FilePath.readFromTar(FilePath.java:2071)
      	...
      Caused by: java.io.IOException: incorrect header check
      	at com.jcraft.jzlib.InflaterInputStream.read(InflaterInputStream.java:112)
      	at org.apache.tools.tar.TarBuffer.readBlock(TarBuffer.java:257)
      	...
      

      I will try and fix the tool installer in question to resolve the end URL, but this isn't a great idea in the long run — Jenkins should follow the 30x redirects itself.

          [JENKINS-23507] FilePath.installIfNecessaryFrom does not follow HTTP 30x redirects

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: 15knots Path: cmake.groovy http://jenkins-ci.org/commit/backend-crawler/9e81b09be1e950d2feed836fc5fe9bb0ae92ad00 Log: Workarounbd for JENKINS-23507 Compare: https://github.com/jenkinsci/backend-crawler/compare/2dd4c0054441...9e81b09be1e9

          Vincent Latombe added a comment - Filed https://github.com/jenkinsci/jenkins/pull/2331 to fix this issue.

          Code changed in jenkins
          User: Vincent Latombe
          Path:
          core/src/main/java/hudson/FilePath.java
          core/src/test/java/hudson/FilePathTest.java
          http://jenkins-ci.org/commit/jenkins/23a37feec061c4def89b1ae65976b55ae7bc3be4
          Log:
          JENKINS-23507 Make FilePath.installIfNecessaryFrom follow redirects

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Vincent Latombe Path: core/src/main/java/hudson/FilePath.java core/src/test/java/hudson/FilePathTest.java http://jenkins-ci.org/commit/jenkins/23a37feec061c4def89b1ae65976b55ae7bc3be4 Log: JENKINS-23507 Make FilePath.installIfNecessaryFrom follow redirects

          Code changed in jenkins
          User: Vincent Latombe
          Path:
          core/src/main/java/hudson/FilePath.java
          core/src/test/java/hudson/FilePathTest.java
          http://jenkins-ci.org/commit/jenkins/0c16c8576bf662b2af049d8d9720ea7d30085c3a
          Log:
          JENKINS-23507 Make FilePath.installIfNecessaryFrom follow redirects (#2331)

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Vincent Latombe Path: core/src/main/java/hudson/FilePath.java core/src/test/java/hudson/FilePathTest.java http://jenkins-ci.org/commit/jenkins/0c16c8576bf662b2af049d8d9720ea7d30085c3a Log: JENKINS-23507 Make FilePath.installIfNecessaryFrom follow redirects (#2331)

          Oleg Nenashev added a comment -

          Released as Jenkins-2.4

          Oleg Nenashev added a comment - Released as Jenkins-2.4

          Carlos Melo added a comment -

          oleg_nenashev is this issue back?

          I just got the following on jenkins 2.15:

          Unpacking Go from https://storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz to /Users/carlosmelo/.jenkins/tools/org.jenkinsci.plugins.golang.GolangInstallation/Go1.5.4 on Jenkins
          FATAL: Failed to install https://storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz to /Users/carlosmelo/.jenkins/tools/org.jenkinsci.plugins.golang.GolangInstallation/Go1.5.4
          java.io.IOException: Failed to install https://storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz to /Users/carlosmelo/.jenkins/tools/org.jenkinsci.plugins.golang.GolangInstallation/Go1.5.4
          	at hudson.FilePath.installIfNecessaryFrom(FilePath.java:860)
          	at hudson.FilePath.installIfNecessaryFrom(FilePath.java:768)
          	at org.jenkinsci.plugins.golang.GolangInstaller.performInstallation(GolangInstaller.java:56)
          	at hudson.tools.InstallerTranslator.getToolHome(InstallerTranslator.java:68)
          	at hudson.tools.ToolLocationNodeProperty.getToolHome(ToolLocationNodeProperty.java:109)
          	at hudson.tools.ToolInstallation.translateFor(ToolInstallation.java:206)
          	at org.jenkinsci.plugins.golang.GolangInstallation.forNode(GolangInstallation.java:43)
          	at org.jenkinsci.plugins.golang.GolangBuildWrapper.setUp(GolangBuildWrapper.java:34)
          	at hudson.model.Build$BuildExecution.doRun(Build.java:156)
          	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534)
          	at hudson.model.Run.execute(Run.java:1720)
          	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
          	at hudson.model.ResourceController.execute(ResourceController.java:98)
          	at hudson.model.Executor.run(Executor.java:404)
          Caused by: java.io.IOException: Failed to unpack https://storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz (152490 bytes read of total 80053651)
          	at hudson.FilePath.installIfNecessaryFrom(FilePath.java:854)
          	... 13 more
          Caused by: java.io.IOException: Failed to extract input stream
          	at hudson.FilePath.readFromTar(FilePath.java:2328)
          	at hudson.FilePath.access$400(FilePath.java:195)
          	at hudson.FilePath$10.invoke(FilePath.java:730)
          	at hudson.FilePath$10.invoke(FilePath.java:728)
          	at hudson.FilePath.act(FilePath.java:1018)
          	at hudson.FilePath.act(FilePath.java:996)
          	at hudson.FilePath.untarFrom(FilePath.java:728)
          	at hudson.FilePath.installIfNecessaryFrom(FilePath.java:852)
          	... 13 more
          Caused by: java.io.IOException: Truncated TAR archive
          	at org.apache.commons.compress.archivers.tar.TarArchiveInputStream.read(TarArchiveInputStream.java:618)
          	at java.io.InputStream.read(InputStream.java:101)
          	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1792)
          	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769)
          	at org.apache.commons.io.IOUtils.copy(IOUtils.java:1744)
          	at hudson.util.IOUtils.copy(IOUtils.java:40)
          	at hudson.FilePath.readFromTar(FilePath.java:2318)
          	... 20 more
          

          Carlos Melo added a comment - oleg_nenashev is this issue back? I just got the following on jenkins 2.15: Unpacking Go from https: //storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz to /Users/carlosmelo/.jenkins/tools/org.jenkinsci.plugins.golang.GolangInstallation/Go1.5.4 on Jenkins FATAL: Failed to install https: //storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz to /Users/carlosmelo/.jenkins/tools/org.jenkinsci.plugins.golang.GolangInstallation/Go1.5.4 java.io.IOException: Failed to install https: //storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz to /Users/carlosmelo/.jenkins/tools/org.jenkinsci.plugins.golang.GolangInstallation/Go1.5.4 at hudson.FilePath.installIfNecessaryFrom(FilePath.java:860) at hudson.FilePath.installIfNecessaryFrom(FilePath.java:768) at org.jenkinsci.plugins.golang.GolangInstaller.performInstallation(GolangInstaller.java:56) at hudson.tools.InstallerTranslator.getToolHome(InstallerTranslator.java:68) at hudson.tools.ToolLocationNodeProperty.getToolHome(ToolLocationNodeProperty.java:109) at hudson.tools.ToolInstallation.translateFor(ToolInstallation.java:206) at org.jenkinsci.plugins.golang.GolangInstallation.forNode(GolangInstallation.java:43) at org.jenkinsci.plugins.golang.GolangBuildWrapper.setUp(GolangBuildWrapper.java:34) at hudson.model.Build$BuildExecution.doRun(Build.java:156) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534) at hudson.model.Run.execute(Run.java:1720) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:98) at hudson.model.Executor.run(Executor.java:404) Caused by: java.io.IOException: Failed to unpack https: //storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz (152490 bytes read of total 80053651) at hudson.FilePath.installIfNecessaryFrom(FilePath.java:854) ... 13 more Caused by: java.io.IOException: Failed to extract input stream at hudson.FilePath.readFromTar(FilePath.java:2328) at hudson.FilePath.access$400(FilePath.java:195) at hudson.FilePath$10.invoke(FilePath.java:730) at hudson.FilePath$10.invoke(FilePath.java:728) at hudson.FilePath.act(FilePath.java:1018) at hudson.FilePath.act(FilePath.java:996) at hudson.FilePath.untarFrom(FilePath.java:728) at hudson.FilePath.installIfNecessaryFrom(FilePath.java:852) ... 13 more Caused by: java.io.IOException: Truncated TAR archive at org.apache.commons.compress.archivers.tar.TarArchiveInputStream.read(TarArchiveInputStream.java:618) at java.io.InputStream.read(InputStream.java:101) at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1792) at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769) at org.apache.commons.io.IOUtils.copy(IOUtils.java:1744) at hudson.util.IOUtils.copy(IOUtils.java:40) at hudson.FilePath.readFromTar(FilePath.java:2318) ... 20 more

          cmelo

          Looks to me the url is resolved properly. I get an HTTP 200 from curl -I https://storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz

          So whatever is your problem, it's probably unrelated to this issue.

          Vincent Latombe added a comment - cmelo Looks to me the url is resolved properly. I get an HTTP 200 from curl -I https://storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz So whatever is your problem, it's probably unrelated to this issue.

            vlatombe Vincent Latombe
            orrc Christopher Orr
            Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: