[JENKINS-23507] FilePath.installIfNecessaryFrom does not follow HTTP 30x redirects - Jenkins Jira

XML

Word

Printable

Details

Type: Bug
Status: Resolved (View Workflow)
Priority: Major
Resolution: Fixed
Component/s: core
Labels:
None

Similar Issues:

Show

Description

I'm using a tool installer which, for some downloads, the URL returns a 302 redirect.

However, Jenkins reads the contents of the redirect page (i.e. some text like "This page has moved to http://foo/bar") and attempts to parse that as the zip/tar/whatever file.

The stacktrace users see when this happens is pretty cryptic:

Unpacking Go from http://golang.org/dl/go1.3.darwin-amd64-osx10.6.tar.gz to /Users/Shared/Jenkins/Home/tools/org.jenkinsci.plugins.golang.GolangInstallation/Go_1.3 on Jenkins
FATAL: Failed to install http://golang.org/dl/go1.3.darwin-amd64-osx10.6.tar.gz to /Users/Shared/Jenkins/Home/tools/org.jenkinsci.plugins.golang.GolangInstallation/Go_1.3
hudson.util.IOException2: Failed to install http://golang.org/dl/go1.3.darwin-amd64-osx10.6.tar.gz to /Users/Shared/Jenkins/Home/tools/org.jenkinsci.plugins.golang.GolangInstallation/Go_1.3
	at hudson.FilePath.installIfNecessaryFrom(FilePath.java:768)
	at org.jenkinsci.plugins.golang.GolangInstaller.performInstallation(GolangInstaller.java:53)
	at hudson.tools.InstallerTranslator.getToolHome(InstallerTranslator.java:61)
	at hudson.tools.ToolLocationNodeProperty.getToolHome(ToolLocationNodeProperty.java:107)
	at hudson.tools.ToolInstallation.translateFor(ToolInstallation.java:204)
	at org.jenkinsci.plugins.golang.GolangInstallation.forNode(GolangInstallation.java:41)
	at org.jenkinsci.plugins.golang.GolangBuildWrapper.setUp(GolangBuildWrapper.java:34)
        ...
Caused by: hudson.util.IOException2: Failed to unpack http://golang.org/dl/go1.3.darwin-amd64-osx10.6.tar.gz (28 bytes read of total -1)
	at hudson.FilePath.installIfNecessaryFrom(FilePath.java:762)
	... 12 more
Caused by: hudson.util.IOException2: Failed to extract input stream
	at hudson.FilePath.readFromTar(FilePath.java:2071)
	...
Caused by: java.io.IOException: incorrect header check
	at com.jcraft.jzlib.InflaterInputStream.read(InflaterInputStream.java:112)
	at org.apache.tools.tar.TarBuffer.readBlock(TarBuffer.java:257)
	...

I will try and fix the tool installer in question to resolve the end URL, but this isn't a great idea in the long run — Jenkins should follow the 30x redirects itself.

Attachments

Issue Links

is related to

JENKINS-23509 Go versions newer than 1.2.1 cannot be installed

Closed

links to

CloudBees Internal CJP-4658

Activity

Ascending order - Click to sort in descending order

SCM/JIRA link daemon added a comment - 2015-11-05 18:33

Code changed in jenkins
User: 15knots
Path:
cmake.groovy
http://jenkins-ci.org/commit/backend-crawler/9e81b09be1e950d2feed836fc5fe9bb0ae92ad00
Log:
Workarounbd for ~~JENKINS-23507~~

Compare: https://github.com/jenkinsci/backend-crawler/compare/2dd4c0054441...9e81b09be1e9

SCM/JIRA link daemon added a comment - 2015-11-05 18:33 Code changed in jenkins User: 15knots Path: cmake.groovy http://jenkins-ci.org/commit/backend-crawler/9e81b09be1e950d2feed836fc5fe9bb0ae92ad00 Log: Workarounbd for JENKINS-23507 Compare: https://github.com/jenkinsci/backend-crawler/compare/2dd4c0054441...9e81b09be1e9

Vincent Latombe added a comment - 2016-05-11 14:05

Filed https://github.com/jenkinsci/jenkins/pull/2331 to fix this issue.

Vincent Latombe added a comment - 2016-05-11 14:05 Filed https://github.com/jenkinsci/jenkins/pull/2331 to fix this issue.

SCM/JIRA link daemon added a comment - 2016-05-11 15:00

Code changed in jenkins
User: Vincent Latombe
Path:
core/src/main/java/hudson/FilePath.java
core/src/test/java/hudson/FilePathTest.java
http://jenkins-ci.org/commit/jenkins/23a37feec061c4def89b1ae65976b55ae7bc3be4
Log:
~~JENKINS-23507~~ Make FilePath.installIfNecessaryFrom follow redirects

SCM/JIRA link daemon added a comment - 2016-05-11 15:00 Code changed in jenkins User: Vincent Latombe Path: core/src/main/java/hudson/FilePath.java core/src/test/java/hudson/FilePathTest.java http://jenkins-ci.org/commit/jenkins/23a37feec061c4def89b1ae65976b55ae7bc3be4 Log: JENKINS-23507 Make FilePath.installIfNecessaryFrom follow redirects

SCM/JIRA link daemon added a comment - 2016-05-13 10:37

Code changed in jenkins
User: Vincent Latombe
Path:
core/src/main/java/hudson/FilePath.java
core/src/test/java/hudson/FilePathTest.java
http://jenkins-ci.org/commit/jenkins/0c16c8576bf662b2af049d8d9720ea7d30085c3a
Log:
~~JENKINS-23507~~ Make FilePath.installIfNecessaryFrom follow redirects (#2331)

SCM/JIRA link daemon added a comment - 2016-05-13 10:37 Code changed in jenkins User: Vincent Latombe Path: core/src/main/java/hudson/FilePath.java core/src/test/java/hudson/FilePathTest.java http://jenkins-ci.org/commit/jenkins/0c16c8576bf662b2af049d8d9720ea7d30085c3a Log: JENKINS-23507 Make FilePath.installIfNecessaryFrom follow redirects (#2331)

Oleg Nenashev added a comment - 2016-06-06 09:39

Released as Jenkins-2.4

Oleg Nenashev added a comment - 2016-06-06 09:39 Released as Jenkins-2.4

Carlos Melo added a comment - 2016-07-27 18:02

oleg_nenashev is this issue back?

I just got the following on jenkins 2.15:

Unpacking Go from https://storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz to /Users/carlosmelo/.jenkins/tools/org.jenkinsci.plugins.golang.GolangInstallation/Go1.5.4 on Jenkins
FATAL: Failed to install https://storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz to /Users/carlosmelo/.jenkins/tools/org.jenkinsci.plugins.golang.GolangInstallation/Go1.5.4
java.io.IOException: Failed to install https://storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz to /Users/carlosmelo/.jenkins/tools/org.jenkinsci.plugins.golang.GolangInstallation/Go1.5.4
	at hudson.FilePath.installIfNecessaryFrom(FilePath.java:860)
	at hudson.FilePath.installIfNecessaryFrom(FilePath.java:768)
	at org.jenkinsci.plugins.golang.GolangInstaller.performInstallation(GolangInstaller.java:56)
	at hudson.tools.InstallerTranslator.getToolHome(InstallerTranslator.java:68)
	at hudson.tools.ToolLocationNodeProperty.getToolHome(ToolLocationNodeProperty.java:109)
	at hudson.tools.ToolInstallation.translateFor(ToolInstallation.java:206)
	at org.jenkinsci.plugins.golang.GolangInstallation.forNode(GolangInstallation.java:43)
	at org.jenkinsci.plugins.golang.GolangBuildWrapper.setUp(GolangBuildWrapper.java:34)
	at hudson.model.Build$BuildExecution.doRun(Build.java:156)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534)
	at hudson.model.Run.execute(Run.java:1720)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:98)
	at hudson.model.Executor.run(Executor.java:404)
Caused by: java.io.IOException: Failed to unpack https://storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz (152490 bytes read of total 80053651)
	at hudson.FilePath.installIfNecessaryFrom(FilePath.java:854)
	... 13 more
Caused by: java.io.IOException: Failed to extract input stream
	at hudson.FilePath.readFromTar(FilePath.java:2328)
	at hudson.FilePath.access$400(FilePath.java:195)
	at hudson.FilePath$10.invoke(FilePath.java:730)
	at hudson.FilePath$10.invoke(FilePath.java:728)
	at hudson.FilePath.act(FilePath.java:1018)
	at hudson.FilePath.act(FilePath.java:996)
	at hudson.FilePath.untarFrom(FilePath.java:728)
	at hudson.FilePath.installIfNecessaryFrom(FilePath.java:852)
	... 13 more
Caused by: java.io.IOException: Truncated TAR archive
	at org.apache.commons.compress.archivers.tar.TarArchiveInputStream.read(TarArchiveInputStream.java:618)
	at java.io.InputStream.read(InputStream.java:101)
	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1792)
	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769)
	at org.apache.commons.io.IOUtils.copy(IOUtils.java:1744)
	at hudson.util.IOUtils.copy(IOUtils.java:40)
	at hudson.FilePath.readFromTar(FilePath.java:2318)
	... 20 more

Carlos Melo added a comment - 2016-07-27 18:02 oleg_nenashev is this issue back? I just got the following on jenkins 2.15: Unpacking Go from https: //storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz to /Users/carlosmelo/.jenkins/tools/org.jenkinsci.plugins.golang.GolangInstallation/Go1.5.4 on Jenkins FATAL: Failed to install https: //storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz to /Users/carlosmelo/.jenkins/tools/org.jenkinsci.plugins.golang.GolangInstallation/Go1.5.4 java.io.IOException: Failed to install https: //storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz to /Users/carlosmelo/.jenkins/tools/org.jenkinsci.plugins.golang.GolangInstallation/Go1.5.4 at hudson.FilePath.installIfNecessaryFrom(FilePath.java:860) at hudson.FilePath.installIfNecessaryFrom(FilePath.java:768) at org.jenkinsci.plugins.golang.GolangInstaller.performInstallation(GolangInstaller.java:56) at hudson.tools.InstallerTranslator.getToolHome(InstallerTranslator.java:68) at hudson.tools.ToolLocationNodeProperty.getToolHome(ToolLocationNodeProperty.java:109) at hudson.tools.ToolInstallation.translateFor(ToolInstallation.java:206) at org.jenkinsci.plugins.golang.GolangInstallation.forNode(GolangInstallation.java:43) at org.jenkinsci.plugins.golang.GolangBuildWrapper.setUp(GolangBuildWrapper.java:34) at hudson.model.Build$BuildExecution.doRun(Build.java:156) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534) at hudson.model.Run.execute(Run.java:1720) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:98) at hudson.model.Executor.run(Executor.java:404) Caused by: java.io.IOException: Failed to unpack https: //storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz (152490 bytes read of total 80053651) at hudson.FilePath.installIfNecessaryFrom(FilePath.java:854) ... 13 more Caused by: java.io.IOException: Failed to extract input stream at hudson.FilePath.readFromTar(FilePath.java:2328) at hudson.FilePath.access$400(FilePath.java:195) at hudson.FilePath$10.invoke(FilePath.java:730) at hudson.FilePath$10.invoke(FilePath.java:728) at hudson.FilePath.act(FilePath.java:1018) at hudson.FilePath.act(FilePath.java:996) at hudson.FilePath.untarFrom(FilePath.java:728) at hudson.FilePath.installIfNecessaryFrom(FilePath.java:852) ... 13 more Caused by: java.io.IOException: Truncated TAR archive at org.apache.commons.compress.archivers.tar.TarArchiveInputStream.read(TarArchiveInputStream.java:618) at java.io.InputStream.read(InputStream.java:101) at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1792) at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769) at org.apache.commons.io.IOUtils.copy(IOUtils.java:1744) at hudson.util.IOUtils.copy(IOUtils.java:40) at hudson.FilePath.readFromTar(FilePath.java:2318) ... 20 more

Vincent Latombe added a comment - 2016-07-27 18:22

cmelo

Looks to me the url is resolved properly. I get an HTTP 200 from curl -I https://storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz

So whatever is your problem, it's probably unrelated to this issue.

Vincent Latombe added a comment - 2016-07-27 18:22 cmelo Looks to me the url is resolved properly. I get an HTTP 200 from curl -I https://storage.googleapis.com/golang/go1.5.4.darwin-amd64.tar.gz So whatever is your problem, it's probably unrelated to this issue.

People

Assignee:: Vincent Latombe

Reporter:: Christopher Orr

Votes:: 1 Vote for this issue

Watchers:: 5 Start watching this issue

Dates

Created:: 2014-06-20 10:17

Updated:: 2017-12-07 19:11

Resolved:: 2016-05-13 10:40