-
Bug
-
Resolution: Fixed
-
Critical
-
Debian 2.6.32-5-amd
-
Powered by SuggestiMate
Hi,
We have encountered problems when using File parameter in our jobs.
How to reproduce problem:
1) create job
2) mark "This build is parametrized"
2a) Add Parameter - File Parameter
2b) File location - "data.file"
2c) description - "plik"
3) Build
3a) Add build step - Execute shell:
ls -la
mv data.file data.zip
sleep 10
rm *
4) trigger new job and use big *.zip file as parameter (~150MB)
During file upload on master in /tmp directory new file will be created - for example "upload_6e074d3b_136c03af218__8000_00000013.tmp".
When job finish this *.tmp file usually is deleted automatically - and here our problems are starting.
In build.xml additional note is added:
<file class="org.apache.commons.fileupload.disk.DiskFileItem" serialization="custom">
<org.apache.commons.fileupload.disk.DiskFileItem>
<default>
<isFormField>false</isFormField>
<size>153052917</size>
<sizeThreshold>10240</sizeThreshold>
<contentType>application/zip</contentType>
<dfosFile>/tmp/upload_6e074d3b_136c03af218__8000_00000013.tmp</dfosFile>
<fieldName>file0</fieldName>
<fileName>WN6.0_MP4.4_22.12.zip</fileName>
</default>
</org.apache.commons.fileupload.disk.DiskFileItem>
</file>
When we are restarting jenkins it is not able to read this jobs because this temporary file do not exists - job history is not visible from Jenkins but builds exists on machine. Error message after Jenkins restart:
...
Caused by: com.thoughtworks.xstream.converters.ConversionException: Could not call org.apache.commons.fileupload.disk.DiskFileItem.readObject() : /tmp/upload_6e074d3b_136c03af218__8000_00000013.tmp (No such file or directory)
---- Debugging information ----
message : Could not call org.apache.commons.fileupload.disk.DiskFileItem.readObject()
cause-exception : java.io.FileNotFoundException
cause-message : /tmp/upload_6e074d3b_136c03af218__8000_00000013.tmp (No such file or directory)
class : hudson.model.FreeStyleBuild
required-type : org.apache.commons.fileupload.disk.DiskFileItem
path : /build/actions/hudson.model.ParametersAction/parameters/hudson.model.FileParameterValue/file/org.apache.commons.fileupload.disk.DiskFileItem
line number : 32
-------------------------------
...
Could you please help to check this?
- is duplicated by
-
JENKINS-12219 Missing builds after restart due to error loading FileParameters
-
- Resolved
-
-
JENKINS-11938 Jenkins loses builds when restarted
-
- Closed
-
- is related to
-
JENKINS-21024 Miscellaneous exceptions in config.xml can prevent entire job from loading
-
- Resolved
-
[JENKINS-13536] File parameter causing data lost after Jenkins restart
Have noticed that the following 'hack' on the offending build.xml file will allow a 'Reload configuration from disk' operation to parse and load the missing build in the UI. Simply replace the offending <dfosFile> element with an empty <cachedContent> element.
So in the original example above:
replace
<dfosFile>/tmp/upload_6e074d3b_136c03af218__8000_00000013.tmp</dfosFile>
with
<cachedContent></cachedContent>
Reproduced more simply:
- create job
- add file parameter, name irrelevant
- build job
- specify any file
- shut down Jenkins
- see that build.xml contains a filename in /tmp; delete it
- restart Jenkins
Have not figured out how to reproduce in a functional test. Starting with
ParametersDefinitionProperty pdp = new ParametersDef FreeStyleProject p = j.createFreeStyleProject(); p.addProperty(pdp); p.save(); File f = new File(j.createTmpDir(), "stuff"); FileUtils.writeStringToFile(f, "some stuff here\n");
it is not clear how to proceed.
p.scheduleBuild2(0, null, new ParametersAction(new FileParameterValue("stuff", f, "stuff")));
is the obvious approach but this would be using FileParameterValue.FileItemImpl which bypasses the actual bug.
Using HtmlUnit (WebRequestSettings, HtmlPage, HtmlForm, HtmlFileInput) would closely follow the user workflow but loading the initial form fails due to crumb issues.
curl -F file=@/tmp/… -Fjson='{"parameter": {"name": "file", "file": "file"}}' 'http://localhost:8080/job/…/build?delay=0sec'
works from the command line but HttpUnit is not currently available in jenkins/test so there is no apparent way to directly simulate this.
For now will just have to verify fix manually.
Integrated in jenkins_main_trunk #2121
[FIXED JENKINS-13536] Using file parameters could cause build records to not load. (Revision 4dde24e62037439b7b73addd7cefae83a254eb3c)
Result = SUCCESS
Jesse Glick : 4dde24e62037439b7b73addd7cefae83a254eb3c
Files :
- changelog.html
- core/src/main/java/hudson/model/FileParameterValue.java
Code changed in jenkins
User: Jesse Glick
Path:
changelog.html
core/src/main/java/hudson/model/FileParameterValue.java
http://jenkins-ci.org/commit/jenkins/4dde24e62037439b7b73addd7cefae83a254eb3c
Log:
[FIXED JENKINS-13536] Using file parameters could cause build records to not load.
Solves the issue for newly created builds by not attempting to persist the original file upload record, which is useless after the build has run (the fileParameters dir in the build already archives them).
Does not help loading pre-fix builds in which the upload temp file has been deleted; for these, you must manually delete <file class="org.apache.commons.fileupload.disk.DiskFileItem" serialization="custom">…</file> from the build.xml.
(Did not find a straightforward way to just instruct XStream to ignore this bit of XML: it is easy to avoid serializing a field, but deserializing is different.)
This still fails for me in 1.494
Dec 17, 2012 9:26:59 PM hudson.model.RunMap retrieve
WARNING: could not load /vol/ebs1/jenkins/jobs/ah-trunk-hardy/builds/2012-10-17_21-17-23
hudson.util.IOException2: Unable to read /vol/ebs1/jenkins/jobs/ah-trunk-hardy/builds/2012-10-17_21-17-23/build.xml
at hudson.XmlFile.unmarshal(XmlFile.java:164)
at hudson.model.Run.reload(Run.java:303)
at hudson.model.Run.<init>(Run.java:292)
at hudson.model.AbstractBuild.<init>(AbstractBuild.java:179)
at hudson.model.Build.<init>(Build.java:103)
at hudson.model.FreeStyleBuild.<init>(FreeStyleBuild.java:41)
at sun.reflect.GeneratedConstructorAccessor40.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
at hudson.model.AbstractProject.loadBuild(AbstractProject.java:1078)
at hudson.model.AbstractProject$1.create(AbstractProject.java:276)
at hudson.model.AbstractProject$1.create(AbstractProject.java:274)
at hudson.model.RunMap.retrieve(RunMap.java:220)
at hudson.model.RunMap.retrieve(RunMap.java:59)
at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:638)
at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:621)
at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:432)
at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:207)
at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:100)
at hudson.model.Job.getLastSuccessfulBuild(Job.java:770)
at sun.reflect.GeneratedMethodAccessor45.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.apache.commons.jexl.util.PropertyExecutor.execute(PropertyExecutor.java:125)
at org.apache.commons.jexl.util.introspection.UberspectImpl$VelGetterImpl.invoke(UberspectImpl.java:314)
at org.apache.commons.jexl.parser.ASTArrayAccess.evaluateExpr(ASTArrayAccess.java:185)
at org.apache.commons.jexl.parser.ASTIdentifier.execute(ASTIdentifier.java:75)
at org.apache.commons.jexl.parser.ASTReference.execute(ASTReference.java:83)
at org.apache.commons.jexl.parser.ASTReference.value(ASTReference.java:57)
at org.apache.commons.jexl.parser.ASTReferenceExpression.value(ASTReferenceExpression.java:51)
at org.apache.commons.jexl.ExpressionImpl.evaluate(ExpressionImpl.java:80)
at hudson.ExpressionFactory2$JexlExpression.evaluate(ExpressionFactory2.java:72)
at org.apache.commons.jelly.tags.core.CoreTagLibrary$3.run(CoreTagLibrary.java:134)
at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:81)
at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:146)
at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
at org.apache.commons.jelly.tags.core.ForEachTag.doTag(ForEachTag.java:150)
at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
at org.apache.commons.jelly.tags.core.ForEachTag.doTag(ForEachTag.java:150)
at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
at org.apache.commons.jelly.tags.core.CoreTagLibrary$1.run(CoreTagLibrary.java:98)
at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
at org.kohsuke.stapler.jelly.groovy.JellyBuilder.doInvokeMethod(JellyBuilder.java:283)
at org.kohsuke.stapler.jelly.groovy.Namespace$ProxyImpl.invoke(Namespace.java:92)
at $Proxy40.projectView(Unknown Source)
at lib.JenkinsTagLib$projectView.call(Unknown Source)
at hudson.model.View.main.run(main.groovy:14)
at org.kohsuke.stapler.jelly.groovy.GroovierJellyScript.run(GroovierJellyScript.java:66)
at org.kohsuke.stapler.jelly.groovy.GroovierJellyScript.run(GroovierJellyScript.java:59)
at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:146)
at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98)
at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
at org.apache.commons.jelly.tags.core.CoreTagLibrary$1.run(CoreTagLibrary.java:98)
at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98)
at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
at org.kohsuke.stapler.jelly.CompressTag.doTag(CompressTag.java:44)
at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:81)
at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:63)
at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:53)
at org.kohsuke.stapler.jelly.JellyClassTearOff.serveIndexJelly(JellyClassTearOff.java:107)
at org.kohsuke.stapler.jelly.JellyFacet.handleIndexRequest(JellyFacet.java:127)
at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:562)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:658)
at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:624)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:658)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:487)
at org.kohsuke.stapler.Stapler.service(Stapler.java:164)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:45)
at winstone.ServletConfiguration.execute(ServletConfiguration.java:248)
at winstone.RequestDispatcher.forward(RequestDispatcher.java:333)
at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:376)
at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:95)
at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:87)
at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47)
at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:166)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:173)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at jenkins.security.ApiTokenFilter.doFilter(ApiTokenFilter.java:63)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:66)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)
at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:50)
at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
at winstone.RequestDispatcher.forward(RequestDispatcher.java:331)
at winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:215)
at winstone.RequestHandlerThread.run(RequestHandlerThread.java:138)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
Caused by: com.thoughtworks.xstream.converters.ConversionException: Could not call org.apache.commons.fileupload.disk.DiskFileItem.readObject() : /tmp/upload_37e81281_13a6b678425_8000_00000068.tmp (No such file or directory)
---- Debugging information ----
message : Could not call org.apache.commons.fileupload.disk.DiskFileItem.readObject()
cause-exception : java.io.FileNotFoundException
cause-message : /tmp/upload_37e81281_13a6b678425_8000_00000068.tmp (No such file or directory)
class : hudson.model.FreeStyleBuild
required-type : org.apache.commons.fileupload.disk.DiskFileItem
path : /build/actions/hudson.model.ParametersAction/parameters/hudson.model.FileParameterValue/file/org.apache.commons.fileupload.disk.DiskFileItem
line number : 44
-------------------------------
As I wrote in the commit comment, the fix does nothing for existing build records—only for newly created ones. For now, old build records need to be manually cleaned up.
We are using the jenkin version 1.487
Could you please explain the how do we show the last build manually?
Why the history is not loaded based on timestamps ?
Code changed in jenkins
User: Jesse Glick
Path:
changelog.html
core/src/main/java/hudson/model/FileParameterValue.java
http://jenkins-ci.org/commit/jenkins/5f7ad6e5feee4c70e2d13d68e80e086abf6a9f50
Log:
[FIXED JENKINS-13536] Using file parameters could cause build records to not load.
Solves the issue for newly created builds by not attempting to persist the original file upload record, which is useless after the build has run (the fileParameters dir in the build already archives them).
Does not help loading pre-fix builds in which the upload temp file has been deleted; for these, you must manually delete <file class="org.apache.commons.fileupload.disk.DiskFileItem" serialization="custom">…</file> from the build.xml.
(Did not find a straightforward way to just instruct XStream to ignore this bit of XML: it is easy to avoid serializing a field, but deserializing is different.)(cherry picked from commit 4dde24e62037439b7b73addd7cefae83a254eb3c)
Conflicts:
changelog.html
Compare: https://github.com/jenkinsci/jenkins/compare/8734bb733dc4...5f7ad6e5feee
Same issue for me. Groovy Parametrized Builds. Working with Windows x64 Jenkins v1.488 (running as a service)