[JENKINS-6962] Unable to delete projects on occasion with jobconfighistory plugin installed when on nfs mount

Type: Bug
Resolution: Fixed
Priority: Major
Component/s: jobconfighistory-plugin
Labels:
None

Similar Issues:
Powered by SuggestiMate

Show

I've run into this a few times, where I can't delete a job due to some file that still has a handle on it in the config-history directory for the job.

java.io.IOException: Unable to delete /opt/hudson/home/jobs/TestProject/config-history/2010-07-13_10-31-57/.nfs000000000012cc65000000f2
	at hudson.Util.deleteFile(Util.java:260)
	at hudson.Util.deleteRecursive(Util.java:302)
	at hudson.Util.deleteContentsRecursive(Util.java:221)
	at hudson.Util.deleteRecursive(Util.java:301)
	at hudson.Util.deleteContentsRecursive(Util.java:221)
	at hudson.Util.deleteRecursive(Util.java:301)
	at hudson.Util.deleteContentsRecursive(Util.java:221)
	at hudson.Util.deleteRecursive(Util.java:301)
	at hudson.model.AbstractItem.performDelete(AbstractItem.java:280)
	at hudson.model.Job.performDelete(Job.java:220)
	at hudson.model.AbstractProject.performDelete(AbstractProject.java:259)
	at hudson.model.AbstractItem.delete(AbstractItem.java:267)
	at hudson.model.AbstractItem.doDoDelete(AbstractItem.java:249)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:235)
	at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:116)
	at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:57)
	at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:75)
	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:30)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:526)
	at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:181)
	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:30)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:526)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:442)
	at org.kohsuke.stapler.Stapler.service(Stapler.java:124)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:45)
	at winstone.ServletConfiguration.execute(ServletConfiguration.java:249)
	at winstone.RequestDispatcher.forward(RequestDispatcher.java:335)
	at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:378)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:94)
	at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:388)
	at org.jvnet.hudson.plugins.monitoring.HudsonMonitoringFilter.doFilter(HudsonMonitoringFilter.java:42)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:97)
	at hudson.plugins.audit_trail.AuditTrailFilter.doFilter(AuditTrailFilter.java:64)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:97)
	at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:86)
	at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
	at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
	at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47)
	at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
	at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
	at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:166)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:173)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
	at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:66)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
	at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)
	at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
	at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
	at winstone.RequestDispatcher.forward(RequestDispatcher.java:333)
	at winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:244)
	at winstone.RequestHandlerThread.run(RequestHandlerThread.java:150)
	at java.lang.Thread.run(Thread.java:619)

is related to

JENKINS-8550 Hudson build fails with unable to delete NFS lock file

Open

Mirko Friedenhagen added a comment - 2010-12-22 01:38

Version 1.9 should have a workaround for this https://github.com/hudson/jobConfigHistory-plugin/commit/50e3f04b378570a51d06c1d54ee4b8c84f92b03c
However deletion, rename etc. do not take actually, you just do not get an error anymore which is kind of hacky.
Best working solution would be to avoid Windows or NFS .

Mirko Friedenhagen added a comment - 2010-12-22 01:38 Version 1.9 should have a workaround for this https://github.com/hudson/jobConfigHistory-plugin/commit/50e3f04b378570a51d06c1d54ee4b8c84f92b03c However deletion, rename etc. do not take actually, you just do not get an error anymore which is kind of hacky. Best working solution would be to avoid Windows or NFS .

Mirko Friedenhagen added a comment - 2010-12-22 01:40

Part of released version 1.9

Mirko Friedenhagen added a comment - 2010-12-22 01:40 Part of released version 1.9

Jacob Robertson added a comment - 2012-02-28 13:50

I'm still seeing this exact error. I'm on version 1.11, on Jenkins LTS 1.409.1, and on Windows Server 2008.
This is a show stopper for us. We came to rely on this plugin, but this bug completely breaks Jenkins. The workaround you describe (and I looked at the git diff) would not be acceptable, because this isn't an intermittent problem, it happens every single time we try and save a job.

Is there any chance that this could actually be fixed? If not, I suggest you change the resolution to "Will Not Fix" rather than "Fixed".

I would be happy to help implement a fix, but I could use a little more background to understand why this particular plugin has this problem, but not others.

Jacob Robertson added a comment - 2012-02-28 13:50 I'm still seeing this exact error. I'm on version 1.11, on Jenkins LTS 1.409.1, and on Windows Server 2008. This is a show stopper for us. We came to rely on this plugin, but this bug completely breaks Jenkins. The workaround you describe (and I looked at the git diff) would not be acceptable, because this isn't an intermittent problem, it happens every single time we try and save a job. Is there any chance that this could actually be fixed? If not, I suggest you change the resolution to "Will Not Fix" rather than "Fixed". I would be happy to help implement a fix, but I could use a little more background to understand why this particular plugin has this problem, but not others.

Jacob Robertson added a comment - 2012-02-28 14:02

Actually, I'm not positive this is the exact same error, as my trace looks different. It had "unable to delete" so I might have jumped to conclusions. Here is the trace that comes up any time we try to save any job's config:

java.io.IOException: Unable to delete H:\jenkins\jobs\testjob\config.xml
at hudson.util.AtomicFileWriter.commit(AtomicFileWriter.java:112)
at hudson.XmlFile.write(XmlFile.java:166)
at hudson.model.AbstractItem.save(AbstractItem.java:368)
at hudson.model.Job.save(Job.java:155)
at hudson.util.PersistedList.onModified(PersistedList.java:152)
at hudson.util.PersistedList.replaceBy(PersistedList.java:79)
at hudson.util.DescribableList.rebuild(DescribableList.java:165)
at hudson.model.Project.submit(Project.java:196)
at hudson.model.FreeStyleProject.submit(FreeStyleProject.java:99)
at hudson.model.Job.doConfigSubmit(Job.java:970)
at hudson.model.AbstractProject.doConfigSubmit(AbstractProject.java:624)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:282)
at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:149)
at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:88)
at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:103)
at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:561)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:646)
at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:233)
at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:561)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:646)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:477)
at org.kohsuke.stapler.Stapler.service(Stapler.java:159)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:45)
at winstone.ServletConfiguration.execute(ServletConfiguration.java:249)
at winstone.RequestDispatcher.forward(RequestDispatcher.java:335)
at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:378)
at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:94)
at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:86)
at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47)
at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)
at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
at winstone.RequestDispatcher.forward(RequestDispatcher.java:333)
at winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:244)
at winstone.RequestHandlerThread.run(RequestHandlerThread.java:150)
at java.lang.Thread.run(Unknown Source)

Jacob Robertson added a comment - 2012-02-28 14:02 Actually, I'm not positive this is the exact same error, as my trace looks different. It had "unable to delete" so I might have jumped to conclusions. Here is the trace that comes up any time we try to save any job's config: java.io.IOException: Unable to delete H:\jenkins\jobs\testjob\config.xml at hudson.util.AtomicFileWriter.commit(AtomicFileWriter.java:112) at hudson.XmlFile.write(XmlFile.java:166) at hudson.model.AbstractItem.save(AbstractItem.java:368) at hudson.model.Job.save(Job.java:155) at hudson.util.PersistedList.onModified(PersistedList.java:152) at hudson.util.PersistedList.replaceBy(PersistedList.java:79) at hudson.util.DescribableList.rebuild(DescribableList.java:165) at hudson.model.Project.submit(Project.java:196) at hudson.model.FreeStyleProject.submit(FreeStyleProject.java:99) at hudson.model.Job.doConfigSubmit(Job.java:970) at hudson.model.AbstractProject.doConfigSubmit(AbstractProject.java:624) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:282) at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:149) at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:88) at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:103) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:561) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:646) at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:233) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:561) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:646) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:477) at org.kohsuke.stapler.Stapler.service(Stapler.java:159) at javax.servlet.http.HttpServlet.service(HttpServlet.java:45) at winstone.ServletConfiguration.execute(ServletConfiguration.java:249) at winstone.RequestDispatcher.forward(RequestDispatcher.java:335) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:378) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:94) at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:86) at winstone.FilterConfiguration.execute(FilterConfiguration.java:195) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368) at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47) at winstone.FilterConfiguration.execute(FilterConfiguration.java:195) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84) at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164) at winstone.FilterConfiguration.execute(FilterConfiguration.java:195) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81) at winstone.FilterConfiguration.execute(FilterConfiguration.java:195) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368) at winstone.RequestDispatcher.forward(RequestDispatcher.java:333) at winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:244) at winstone.RequestHandlerThread.run(RequestHandlerThread.java:150) at java.lang.Thread.run(Unknown Source)

Stefan Brausch added a comment - 2012-02-28 14:15

Maybe is this your problem? http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4073756

Stefan Brausch added a comment - 2012-02-28 14:15 Maybe is this your problem? http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4073756

Jacob Robertson added a comment - 2012-02-28 15:03

I think I just really failed to look closely enough at this bug. I will open a new ticket, because I think it's two separate issues.

Jacob Robertson added a comment - 2012-02-28 15:03 I think I just really failed to look closely enough at this bug. I will open a new ticket, because I think it's two separate issues.

Jacob Robertson added a comment - 2012-02-28 15:04

Putting back to Fixed status. I will open a new ticket.

Jacob Robertson added a comment - 2012-02-28 15:04 Putting back to Fixed status. I will open a new ticket.

Assignee:: Mirko Friedenhagen

Reporter:: Larry Shatzer, Jr.

Votes:: 0 Vote for this issue

Watchers:: 0 Start watching this issue

Created:: 2010-07-13 09:51

Updated:: 2013-04-25 12:22

Resolved:: 2012-02-28 15:04

Details

Description

Attachments

Issue Links

Activity

Collapse comment: Mirko Friedenhagen added a comment - 2010-12-22 01:38

Expand comment: Mirko Friedenhagen added a comment - 2010-12-22 01:38

Collapse comment: Mirko Friedenhagen added a comment - 2010-12-22 01:40

Expand comment: Mirko Friedenhagen added a comment - 2010-12-22 01:40

Collapse comment: Jacob Robertson added a comment - 2012-02-28 13:50

Expand comment: Jacob Robertson added a comment - 2012-02-28 13:50

Collapse comment: Jacob Robertson added a comment - 2012-02-28 14:02

Expand comment: Jacob Robertson added a comment - 2012-02-28 14:02

Collapse comment: Stefan Brausch added a comment - 2012-02-28 14:15

Expand comment: Stefan Brausch added a comment - 2012-02-28 14:15

Collapse comment: Jacob Robertson added a comment - 2012-02-28 15:03

Expand comment: Jacob Robertson added a comment - 2012-02-28 15:03

Collapse comment: Jacob Robertson added a comment - 2012-02-28 15:04

Expand comment: Jacob Robertson added a comment - 2012-02-28 15:04

People

Dates