Is it reproduced with a recent Jenkins version?

No response by report in three months, so closing as incomplete.

Note that the directories with the leading period are created when a build is being deleted, so incomplete deletions don't leave Jenkins-accessible data.

Sorry, I never got the notification from the comment. I'll look into it within the next weeks if it is still occurring on newer versions. Thanks.

current jenkins version 1.563
locked build (keep this forever) doesn't get removed so this particular issue is gone.

however, i've noticed that one build before the locked stays as well instead of getting removed.

removal is set to keep 100 builds
40389
...
40290
40287 (locked)
40285 << should have been removed as 40286, 40288 and 40289 got removed.

robsimon:

Is the build really still there, or is it a stale page load? Reload the browser window. If it's really still there:
Create a new Log Recorder at /log for the logger hudson.tasks.LogRotator on level FINER. Then, run in script console: Jenkins.instance.getItemByFullName('yourjobnamehere').logRotate() and check your logger for what it says prevents deletion of the build.

java.io.IOException: jobname #40285: C:\Jenkins\jobs\jobname\builds\2014-05-19_08-41-18 looks to have already been deleted; siblings: [.2014-03-02_00-41-38, .2014-03-29_04-42-01, .2014-03-31_16-42-06, .2014-04-01_12-42-18, .2014-04-01_20-42-08, .2014-04-12_08-41-59, .2014-04-17_16-42-17, .2014-04-27_00-42-21, .2014-05-08_08-41-39, .2014-05-09_12-41-45, .2014-05-11_04-41-53, .2014-05-19_08-41-18, 2013-12-11_10-01-34, 2014-05-19_16-41-19, 2014-05-20_04-41-03, 2014-05-20_08-41-03, 2014-05-20_12-41-06, 2014-05-20_16-41-05, 2014-05-20_20-41-06, 2014-05-21_00-41-06, 2014-05-21_04-41-07, 2014-05-21_08-41-08, 2014-05-21_12-41-09, 2014-05-21_16-41-10, 2014-05-21_20-41-11, 2014-05-22_00-41-12, 2014-05-22_04-41-12, 2014-05-22_08-41-13, 2014-05-22_12-41-14, 2014-05-22_16-41-14, 2014-05-22_20-41-15, 2014-05-23_00-41-18, 2014-05-23_04-41-17, 2014-05-23_08-41-18, 2014-05-23_12-41-20, 2014-05-23_16-41-19, 2014-05-23_20-41-20, 2014-05-24_03-15-12, 2014-05-24_04-41-22, 2014-05-24_08-41-22, 2014-05-24_12-41-23, 2014-05-24_16-41-25, 2014-05-24_20-41-25, 2014-05-25_00-41-26, 2014-05-25_04-41-26, 2014-05-25_08-41-27, 2014-05-25_12-42-29, 2014-05-25_16-41-29, 2014-05-25_20-41-30, 2014-05-26_00-44-56, 2014-05-26_04-41-31, 2014-05-26_08-41-32, 2014-05-26_12-41-42, 2014-05-26_16-41-34, 2014-05-26_20-41-34, 2014-05-27_00-41-35, 2014-05-27_04-41-36, 2014-05-27_08-41-37, 2014-05-27_12-41-38, 2014-05-27_16-41-39, 2014-05-27_20-41-39, 2014-05-28_00-41-41, 2014-05-28_04-41-41, 2014-05-28_08-41-42, 2014-05-28_12-41-42, 2014-05-28_16-41-44, 2014-05-28_20-41-44, 2014-05-29_00-41-46, 2014-05-29_04-41-46, 2014-05-29_08-41-47, 2014-05-29_12-41-47, 2014-05-29_16-41-49, 2014-05-29_20-41-49, 2014-05-30_00-41-51, 2014-05-30_04-41-51, 2014-05-30_08-41-52, 2014-05-30_12-41-02, 2014-05-30_16-41-02, 2014-05-30_20-41-03, 2014-05-31_00-41-04, 2014-05-31_04-41-05, 2014-05-31_08-41-06, 2014-05-31_12-41-08, 2014-05-31_16-41-08, 2014-05-31_20-41-08, 2014-06-01_00-41-10, 2014-06-01_04-41-10, 2014-06-01_08-41-11, 2014-06-01_12-41-13, 2014-06-01_16-41-13, 2014-06-01_20-41-14, 2014-06-02_00-41-18, 2014-06-02_04-41-16, 2014-06-02_08-41-17, 2014-06-02_12-41-18, 2014-06-02_16-41-19, 2014-06-02_20-41-20, 2014-06-03_00-41-21, 2014-06-03_04-41-22, 2014-06-03_08-41-23, 2014-06-03_11-11-18, 2014-06-03_11-12-17, 2014-06-03_11-13-32, 2014-06-03_11-14-46, 2014-06-03_11-18-58, 2014-06-03_11-19-59, 2014-06-03_11-32-48, 2014-06-03_11-33-34, 2014-06-03_11-35-03, 2014-06-03_11-35-56, 2014-06-03_11-53-03, 2014-06-03_11-53-34, 2014-06-03_12-41-24, 2014-06-03_13-34-21, 32696, 40287, 40290, 40291, 40292, 40293, 40294, 40295, 40296, 40297, 40298, 40299, 40300, 40301, 40302, 40303, 40304, 40305, 40306, 40307, 40308, 40309, 40310, 40311, 40312, 40313, 40314, 40315, 40316, 40317, 40318, 40319, 40320, 40321, 40322, 40323, 40324, 40325, 40326, 40327, 40328, 40329, 40330, 40331, 40332, 40333, 40334, 40335, 40336, 40337, 40338, 40339, 40340, 40341, 40342, 40343, 40344, 40345, 40346, 40347, 40348, 40349, 40350, 40351, 40352, 40353, 40354, 40355, 40356, 40357, 40358, 40359, 40360, 40361, 40362, 40363, 40364, 40365, 40366, 40367, 40368, 40369, 40370, 40371, 40372, 40373, 40374, 40375, 40376, 40377, 40378, 40379, 40380, 40381, 40382, 40383, 40384, 40385, 40386, 40387, 40388, 40389, lastFailedBuild, lastStableBuild, lastSuccessfulBuild, lastUnstableBuild, lastUnsuccessfulBuild]
at hudson.model.Run.delete(Run.java:1454)
at hudson.tasks.LogRotator.perform(LogRotator.java:129)
at hudson.model.Job.logRotate(Job.java:441)
at hudson.model.Job$logRotate.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112)
at Script1.run(Script1.groovy:1)
at groovy.lang.GroovyShell.evaluate(GroovyShell.java:580)
at groovy.lang.GroovyShell.evaluate(GroovyShell.java:618)
at groovy.lang.GroovyShell.evaluate(GroovyShell.java:589)
at hudson.util.RemotingDiagnostics$Script.call(RemotingDiagnostics.java:139)
at hudson.util.RemotingDiagnostics$Script.call(RemotingDiagnostics.java:111)
at hudson.remoting.LocalChannel.call(LocalChannel.java:45)
at hudson.util.RemotingDiagnostics.executeGroovy(RemotingDiagnostics.java:108)
at jenkins.model.Jenkins._doScript(Jenkins.java:3449)
at jenkins.model.Jenkins.doScript(Jenkins.java:3426)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:298)
at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:161)
at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:96)
at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:120)
at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:728)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:858)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:631)
at org.kohsuke.stapler.Stapler.service(Stapler.java:225)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:686)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1494)
at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:96)
at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:202)
at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:180)
at net.bull.javamelody.PluginMonitoringFilter.doFilter(PluginMonitoringFilter.java:85)
at org.jvnet.hudson.plugins.monitoring.HudsonMonitoringFilter.doFilter(HudsonMonitoringFilter.java:90)
at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:99)
at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:58)
at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:99)
at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:88)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:48)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:174)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at jenkins.security.ApiTokenFilter.doFilter(ApiTokenFilter.java:74)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:46)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1474)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:533)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:370)
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:960)
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1021)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:668)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

Jun 03, 2014 2:32:51 PM FINE hudson.tasks.LogRotator
Running the log rotation for hudson.model.FreeStyleProject@338ce0d9[jobname] with numToKeep=100 daysToKeep=-1 artifactNumToKeep=3 artifactDaysToKeep=-1

Jun 03, 2014 2:32:51 PM FINER hudson.tasks.LogRotator
jobname #40287 is not to be removed or purged of artifacts because it’s marked as a keeper

Jun 03, 2014 2:32:51 PM FINE hudson.tasks.LogRotator
jobname #40285 is to be removed

Could you provide some context for the first message with the stack trace?

Also, after running it and getting "jobname #40285 is to be removed", is the build still there in Jenkins? On the disk?

In the regular Jenkins log, there should be an error related to failing to delete files. The '.' gets added when Jenkins removes a build.

2013-12-11_10-01-34 << build directory which isn't visible in UI

Is there a config.xml file in the directory?

What happens to both of these builds when you reload configuration from disk?

1) reloading from disk removed at least #40285 from the Jenkins UI. Why wouldn't it go away by itself without reloading the job configs?

2) 2013-12-11_10-01-34 contains: changelog.xml, log file and timestamper directory which contains timestamps file

3) when will those empty build directories being deleted from HDD which start with . as this job already has 13 of them?

1) reloading from disk removed at least #40285 from the Jenkins UI. Why wouldn't it go away by itself without reloading the job configs?

My guess is that the build deletion got interrupted by some Windows process (e.g. your antivirus or search indexer) locking a file. Hence the build was still in memory, but no longer (in the expected location) on disk, so it could not be loaded again.

2) 2013-12-11_10-01-34 contains: changelog.xml, log file and timestamper directory which contains timestamps file

No build.xml file, so it cannot be loaded. You should see a warning logged about this on Jenkins startup.

3) when will those empty build directories being deleted from HDD which start with . as this job already has 13 of them?

To the best of my knowledge, when you delete them manually. Their existence means the deletion of a build failed.

That a failure to delete the dot-build directory causes Jenkins to keep the build in memory seems weird. Should probably be filed as a bug if it's not already.

@robsimon: those diagnostics were added for JENKINS-22395, though it is not clear there is any relationship.

Hi,
This issue was created in feb, 2013.
I have commented in feb, 2014.
And now about 10 updates from 3 people in 1 week, just for one issue.
What is going on?

sorry for getting side-tracked while going thru this issue here.

i guess this issue can be closed as the original problem of "keep forever builds get removed" does not occur any more with a newer Jenkins version.

evernat: I'm currently going through older issues, closing or poking as appropriate (as I'm sure you've seen, given the number of issues I'm resolving after nobody responded to your comments!). Since I knew Jesse investigated a similar issue as mentioned in a comment ("looks to have already been deleted") I pointed this issue out to him. And finally, the Jira mailer is a bit unreliable, so it's possible your comment wasn't seen by anyone before I posted mine (see robsimon's first comment).

robsimon: I already resolved this as 'Incomplete' after there was no response to evernat's comment – while it seems to be more of a 'Cannot Reproduce', it doesn't really matter. (And we're not consistently closing issues in this Jira AFAICT).

Sorry, false alarm