-
Bug
-
Resolution: Unresolved
-
Major
-
Jenkins 1.598 on CentOS 6.x / tomcat6
-
Powered by SuggestiMate
The zip generated by http://server/job/project/ws/*zip*/workspace.zip truncates at 20.62MB's and results in a corrupt zip archive.
- is related to
-
JENKINS-19752 Download build artifacts as zip generates a corrupted file
-
- Resolved
-
-
JENKINS-19947 Missing base directory in ZIP from .../artifact/dir/subdir/*zip*/subdir.zip
-
- Resolved
-
- links to
[JENKINS-26700] Download build artifacts as zip generates a corrupted file
Seems for me, it affects both sub-directories as well as the entire workspace so long as the file size is > 20.6MB's. Individual artifacts appear to download fine.
I originally noticed this trying to download a sub-dir of the workspace, which should be ~110MB's in zip form, but it always truncates at 20.6MB's and the zip is corrupted (cannot be unzipped).
Seems like it transfers are normal speeds, either 5-10Mbps+ on the local lan (assuming it's not faster due to overhead of creating the zip) and about 1Mbps over the net.
Hi all,
I seem to have the same problem. I'm working on Jenkins 1.581 running on Tomcat 7 on MacOS 10.8.5.
Zipping a subdirectory within the workspace works fine, whereas zipping and download the entire workspace does not. Zipping the entire workspace even creates a smaller zip file (e.g. 80MB) than the zip creation of a subdirectory (115MB)!
Any idea?
It would be great if you could provide a minimal, reproducible test case, such as a shell script build step that generates a bunch of files that result in this problem when downloaded.
Hi all,
I am experiencing this same issue with Jenkins 1.643 running on Linux Debian 8 and the same symptoms as specified by Stefan, the jenkins link "all files in zip" provides a corrupted zip file, our corporate network could be somewhat slow but I don't think that could be related.
Steps to reproduce:
- Create a new freestyle job "test-job", Linux or OS X should be ok
- Add a shell script step with the following content
for i in {1..1000} do dd if=/dev/zero of="ABC1MB-${i}.bin" bs=1024 count=0 seek=1024 done mkdir -p a/b/c/ echo "text" > a/b/c/1 pushd a ln -s b/c/1 2 popd mkdir -p d/e/f pushd d ln -s e/f/1 2 popd
- Run a build to generate workspace
- Download workspace using "(all files in zip)" link in workspace browser
- Try to unzip
Expected results:
- Unzipping will succeed
- Unzipped directory mimics actual workspace, including symlinks
Actual result:
- Invalid archive with error:
# Note: unzip on OS X $ unzip test-job.zip Archive: test-job.zip End-of-central-directory signature not found. Either this file is not a zipfile, or it constitutes one disk of a multi-part archive. In the latter case the central directory and zipfile comment will be found on the last disk(s) of this archive. unzip: cannot find zipfile directory in one of test-job.zip or test-job.zip.zip, and cannot find test-job.zip.ZIP, period. # Note: ditto on OS X $ ditto -x -k test-job.zip . ditto: test-job/d/2: No such file or directory ditto: Couldn't read pkzip signature.
Problems with current state:
- "Broken" symlinks, or symlinks to files that do not exist, apparently cause termination of zip file creation, thereby generating an invalid zip
- "Working" symlinks, or symlinks to files that do exist, add duplicate copies of files
- Recursive symlinks probably cause problems too
Hi all, sadly I'm facing the same issue, here is my environment
jenkins 2.7.4
jdk 1.8.40 (too old?)
centos 6.8 (package updated every night).
It happens that artifacts download (both .zip and other formats) fails often when, in office, the network appears slow. For fail I intend that from browser the download process seems ends correctly ok, but the file downloaded has smaller size and, in case of .zip, can't be opened.
Found this issue in apache's log running on jenkins installation
[Thu Sep 15 13:46:42 2016] [error] [client XXX.XXX.XXX.XXX] (70008)Partial results are valid but processing is incomplete: proxy: error reading response, referer: https://myjenkinsserver/job/mybuild/
I am facing the same issue. We are on Jenkins ver. 2.19.1
We have a powershell script downloading a build artifact zip. The zip is much smaller when the download is complete.
Invoke-WebRequest -URI $jenkinsLink/$majorVersion.$minorVersion/ws/$fileToDownload -ErrorAction:Stop -TimeoutSec 300 -OutFile $fileToDownload
Downloaded zip is unusable, not a valid archive. This is a critical issue and breaks the chain in our CI/Dev Ops.
This is the exception we have in the logs
Caught exception evaluating: request.getSession() in /job/2016.2/ws/PrintStreamUpdate_2016.2.818.0.zip. Reason: java.lang.IllegalStateException: Response is committed
java.lang.IllegalStateException: Response is committed
at org.eclipse.jetty.server.Request.getSession(Request.java:1400)
at org.eclipse.jetty.server.Request.getSession(Request.java:1378)
at javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:279)
at sun.reflect.GeneratedMethodAccessor243.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.commons.jexl.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:258)
at org.apache.commons.jexl.parser.ASTMethod.execute(ASTMethod.java:104)
at org.apache.commons.jexl.parser.ASTReference.execute(ASTReference.java:83)
at org.apache.commons.jexl.parser.ASTReference.value(ASTReference.java:57)
at org.apache.commons.jexl.parser.ASTReferenceExpression.value(ASTReferenceExpression.java:51)
at org.apache.commons.jexl.ExpressionImpl.evaluate(ExpressionImpl.java:80)
at hudson.ExpressionFactory2$JexlExpression.evaluate(ExpressionFactory2.java:74)
at org.apache.commons.jelly.tags.core.CoreTagLibrary$3.run(CoreTagLibrary.java:134)
at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:120)
at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:95)
at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:63)
at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:53)
at org.kohsuke.stapler.jelly.JellyFacet$1.dispatch(JellyFacet.java:95)
at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:649)
at hudson.init.impl.InstallUncaughtExceptionHandler$1.reportException(InstallUncaughtExceptionHandler.java:30)
at org.kohsuke.stapler.compression.CompressionFilter.reportException(CompressionFilter.java:77)
at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:52)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:553)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
I have the same issue with Jenkins 2.19.2 (Windows) while downloading normal artifacts. In this case about 60MB jars. The issue occurs random but quite frequent.
Nov 17, 2016 10:28:16 AM hudson.ExpressionFactory2$JexlExpression evaluate
WARNING: Caught exception evaluating: request.getSession() in /job/<jobname>/artifact/<pathtoartifact>.jar. Reason: java.lang.IllegalStateException: Response is committed
java.lang.IllegalStateException: Response is committed
at org.eclipse.jetty.server.Request.getSession(Request.java:1400)
at org.eclipse.jetty.server.Request.getSession(Request.java:1378)
at javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:279)
at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.commons.jexl.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:258)
at org.apache.commons.jexl.parser.ASTMethod.execute(ASTMethod.java:104)
at org.apache.commons.jexl.parser.ASTReference.execute(ASTReference.java:83)
at org.apache.commons.jexl.parser.ASTReference.value(ASTReference.java:57)
at org.apache.commons.jexl.parser.ASTReferenceExpression.value(ASTReferenceExpression.java:51)
at org.apache.commons.jexl.ExpressionImpl.evaluate(ExpressionImpl.java:80)
at hudson.ExpressionFactory2$JexlExpression.evaluate(ExpressionFactory2.java:74)
at org.apache.commons.jelly.tags.core.CoreTagLibrary$3.run(CoreTagLibrary.java:134)
at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:120)
at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:95)
at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:63)
at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:53)
at org.kohsuke.stapler.jelly.JellyFacet$1.dispatch(JellyFacet.java:95)
at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:649)
at hudson.init.impl.InstallUncaughtExceptionHandler$1.reportException(InstallUncaughtExceptionHandler.java:30)
at org.kohsuke.stapler.compression.CompressionFilter.reportException(CompressionFilter.java:77)
at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:52)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:553)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
On the other end - a Maven Tycho build the problem appears as:
[ERROR] Internal error: org.eclipse.tycho.repository.local.MirroringArtifactProvider$MirroringFailedException: Could not mirror artifact <artifactname> into the local Maven repository.See log output for details. Premature end of Content-Length delimited message body (expected: 77668135; received: 3510470 -> [Help 1]
Guys I bet this is just another broken symlink to be archived. There's a VERY SIMPLE workaround:
Just remove symlinks (If the point outside the archived tree, they would be useless anyway)
I use the following one-liner in my builds and never had ANY problems with zipped artifacts anymore:
find /dir/to/artifacts -type l | xargs rm -f
The following PR should fix this:
https://github.com/jenkinsci/jenkins/pull/2639
I would be very surprised if this would fix Srinivasan Venkataraman's and my problem. In my case it's not with zip files at all.
Should I open a new issue for this?
Well thats a little confusing. After all, the title specifically names zip artifacts. So: yes, I'd suggest a separate issue if it is not about zipped artifacts. Of course thats MY personal opinion only.
I have a similar problem. But it only happens when using the REST API in a job that results in 3 runs. The 2nd run is the one which gets the ZIP file truncated at 75 MB, the actual file size is 99 MB. The file is downloaded immediately after we check the run "result" is "SUCCESS" (not the build). Maybe it's related?
Jenkins 2.289.3
Does this only affect workspace downloads, or also artifact downloads?
What's the transfer speed?