I could reproduce it now on my local machine:
_myjob_trunk #6 main build action completed: SUCCESS
…
SCM changes detected in myjob_trunk. Triggering #8
…
myjob_trunk #7 main build action completed: FAILURE
SCM changes detected in myjob_trunk. Triggering #8
…
Nov 17, 2015 12:27:23 PM SCHWERWIEGEND hudson.model.Executor run
Unexpected executor death
java.lang.IllegalStateException: D:\jenkins\jobs\myjob_trunk\builds\8 already existed; will not overwrite with myjob_trunk #8
at hudson.model.RunMap.put(RunMap.java:189)
at jenkins.model.lazy.LazyBuildMixIn.newBuild(LazyBuildMixIn.java:178)
at hudson.model.AbstractProject.newBuild(AbstractProject.java:1019)
at hudson.model.AbstractProject.createExecutable(AbstractProject.java:1209)
at hudson.model.AbstractProject.createExecutable(AbstractProject.java:145)
at hudson.model.Executor$1.call(Executor.java:362)
at hudson.model.Executor$1.call(Executor.java:344)
at hudson.model.Queue._withLock(Queue.java:1345)
at hudson.model.Queue.withLock(Queue.java:1210)
at hudson.model.Executor.run(Executor.java:344)
Nov 17, 2015 12:27:40 PM INFORMATION hudson.model.Run execute
…
Nov 17, 2015 12:28:33 PM INFORMATION hudson.model.Run execute
myjob_trunk #8 main build action completed: FAILURE
Nov 17, 2015 12:34:01 PM INFORMATION hudson.triggers.SCMTrigger$Runner run
…
Nov 17, 2015 12:56:00 PM INFORMATION hudson.triggers.SCMTrigger$Runner run
SCM changes detected in myjob_trunk. Triggering #9
Nov 17, 2015 12:56:01 PM INFORMATION hudson.triggers.SCMTrigger$Runner run
…
Nov 17, 2015 1:02:00 PM INFORMATION hudson.triggers.SCMTrigger$Runner run
SCM changes detected in myjob_trunk. Triggering #10
…
Nov 17, 2015 1:04:14 PM INFORMATION hudson.model.Run execute
myjob_trunk #9 main build action completed: FAILURE
Nov 17, 2015 1:04:29 PM INFORMATION hudson.model.Run execute
AbwertungsGrund #2 main build action completed: SUCCESS
Nov 17, 2015 1:04:29 PM INFORMATION jenkins.InitReactorRunner$1 onAttained
Started initialization
Nov 17, 2015 1:04:29 PM INFORMATION jenkins.InitReactorRunner$1 onAttained
Listed all plugins
Nov 17, 2015 1:04:29 PM INFORMATION jenkins.InitReactorRunner$1 onAttained
Prepared all plugins
Nov 17, 2015 1:04:29 PM INFORMATION jenkins.InitReactorRunner$1 onAttained
Started all plugins
Nov 17, 2015 1:04:29 PM INFORMATION jenkins.InitReactorRunner$1 onAttained
Augmented all extensions
Nov 17, 2015 1:04:29 PM INFORMATION jenkins.InitReactorRunner$1 onAttained
Loaded all jobs
Nov 17, 2015 1:04:29 PM INFORMATION jenkins.InitReactorRunner$1 onAttained
Completed initialization
Nov 17, 2015 1:06:00 PM INFORMATION hudson.triggers.SCMTrigger$Runner run
…
SCM changes detected in myjob_trunk. Triggering #10
Nov 17, 2015 1:06:02 PM INFORMATION hudson.model.Run execute
…
Nov 17, 2015 1:08:28 PM INFORMATION hudson.model.Run execute
myjob_trunk #10 main build action completed: FAILURE
Nov 17, 2015 1:09:23 PM SCHWERWIEGEND hudson.model.Executor run
Unexpected executor death
java.lang.IllegalStateException: D:\jenkins\jobs\myjob_trunk\builds\10 already existed; will not overwrite with myjob_trunk #10
at hudson.model.RunMap.put(RunMap.java:189)
at jenkins.model.lazy.LazyBuildMixIn.newBuild(LazyBuildMixIn.java:178)
at hudson.model.AbstractProject.newBuild(AbstractProject.java:1019)
at hudson.model.AbstractProject.createExecutable(AbstractProject.java:1209)
at hudson.model.AbstractProject.createExecutable(AbstractProject.java:145)
at hudson.model.Executor$1.call(Executor.java:362)
at hudson.model.Executor$1.call(Executor.java:344)
at hudson.model.Queue._withLock(Queue.java:1345)
at hudson.model.Queue.withLock(Queue.java:1210)
at hudson.model.Executor.run(Executor.java:344)
Nov 17, 2015 1:10:02 PM INFORMATION hudson.triggers.SCMTrigger$Runner run
…
Nov 17, 2015 1:38:00 PM INFORMATION hudson.triggers.SCMTrigger$Runner run
SCM changes detected in myjob_trunk. Triggering #11
Nov 17, 2015 1:38:02 PM INFORMATION hudson.triggers.SCMTrigger$Runner run
…
Nov 17, 2015 1:44:47 PM INFORMATION hudson.model.Run execute
myjob_trunk #11 main build action completed: FAILURE
Nov 17, 2015 1:45:03 PM INFORMATION hudson.model.Run execute
AbwertungsGrund #3 main build action completed: SUCCESS
Nov 17, 2015 1:45:03 PM INFORMATION jenkins.InitReactorRunner$1 onAttained
Started initialization
Nov 17, 2015 1:45:03 PM INFORMATION jenkins.InitReactorRunner$1 onAttained
Listed all plugins
Nov 17, 2015 1:45:03 PM INFORMATION jenkins.InitReactorRunner$1 onAttained
Prepared all plugins
Nov 17, 2015 1:45:03 PM INFORMATION jenkins.InitReactorRunner$1 onAttained
Started all plugins
Nov 17, 2015 1:45:03 PM INFORMATION jenkins.InitReactorRunner$1 onAttained
Augmented all extensions
Nov 17, 2015 1:45:03 PM INFORMATION jenkins.InitReactorRunner$1 onAttained
Loaded all jobs
Nov 17, 2015 1:45:03 PM INFORMATION jenkins.InitReactorRunner$1 onAttained
Completed initialization
Nov 17, 2015 1:45:03 PM INFORMATION org.springframework.web.context.support.StaticWebApplicationContext prepareRefresh
Refreshing org.springframework.web.context.support.StaticWebApplicationContext@7d591a4: display name [Root WebApplicationContext]; startup date [Tue Nov 17 13:45:03 CET 2015]; root of context hierarchy
Nov 17, 2015 1:45:03 PM INFORMATION org.springframework.web.context.support.StaticWebApplicationContext obtainFreshBeanFactory
Bean factory for application context [org.springframework.web.context.support.StaticWebApplicationContext@7d591a4]: org.springframework.beans.factory.support.DefaultListableBeanFactory@3aeb7f67
Nov 17, 2015 1:45:03 PM INFORMATION org.springframework.beans.factory.support.DefaultListableBeanFactory preInstantiateSingletons
Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@3aeb7f67: defining beans [initialDirContextFactory,bindAuthenticator,ldapUserSearch,authenticationManager,authoritiesPopulator]; root of factory hierarchy
…
Nov 17, 2015 2:12:00 PM INFORMATION hudson.triggers.SCMTrigger$Runner run
SCM changes detected in myjob_trunk. Triggering #12
Nov 17, 2015 2:12:02 PM INFORMATION hudson.triggers.SCMTrigger$Runner run
…
Nov 17, 2015 2:16:00 PM INFORMATION hudson.triggers.SCMTrigger$Runner run
SCM changes detected in myjob_trunk. Triggering #13
…
Nov 17, 2015 2:18:00 PM INFORMATION hudson.triggers.SCMTrigger$Runner run
SCM changes detected in myjob_trunk. Triggering #13
Nov 17, 2015 2:18:02 PM INFORMATION hudson.triggers.SCMTrigger$Runner run
…
Nov 17, 2015 2:19:30 PM INFORMATION hudson.model.Run execute
myjob_trunk #12 main build action completed: FAILURE
Nov 17, 2015 2:19:42 PM INFORMATION hudson.model.Run execute
AbwertungsGrund #4 main build action completed: SUCCESS
Nov 17, 2015 2:19:42 PM INFORMATION jenkins.InitReactorRunner$1 onAttained
Started initialization
Nov 17, 2015 2:19:42 PM INFORMATION jenkins.InitReactorRunner$1 onAttained
Listed all plugins
Nov 17, 2015 2:19:42 PM INFORMATION jenkins.InitReactorRunner$1 onAttained
Prepared all plugins
Nov 17, 2015 2:19:42 PM INFORMATION jenkins.InitReactorRunner$1 onAttained
Started all plugins
Nov 17, 2015 2:19:42 PM INFORMATION jenkins.InitReactorRunner$1 onAttained
Augmented all extensions
Nov 17, 2015 2:19:42 PM INFORMATION jenkins.InitReactorRunner$1 onAttained
Loaded all jobs
Nov 17, 2015 2:19:42 PM INFORMATION jenkins.InitReactorRunner$1 onAttained
Completed initialization
_
I'm utilizing some jobs (e.g. "AbwertungsGrund") to show the job's reason for not beeing blue
as a comment within the build-history,
As Tomasz Sniatowski mentioned those jobs/skript do "hudson.model.Hudson.instance.doReload()" which means "Reload Configuration from Disk" I suppose. I don't know any other way to force those comments to be seen. To evaluate if this is the cause for those "dead executors" I now removed "hudson.model.Hudson.instance.doReload()" from those scripts...
no more dead executors since then!!
On my old ci-System running with Jenkins 1.596.2 (and more or less the same jobs) I included "hudson.model.Hudson.instance.doReload()" yesterday. No dead executors either.
But within 1.596.2 "hudson.model.Hudson.instance.doReload()" lead to "Out-of-order builds".
("Timestamp inconsistent")
"hudson.model.Hudson.instance.doReload()" or "Reload(ing) configuration from disk" while there are jobs still running lead to those issues.
I did update from 1.596.2 to 1.625.1 and I am having the same issue. Even if I start from scratch (starting Jenkins with just the jobs config.xml in their directories-no other directories or symbolic links present) Build #1 is ok in most cases - but after some builds the error described above appears:
java.lang.IllegalStateException: C:\Jenkins\jobs\xyz_trunk\builds\25 already existed; will not overwrite with xyz_trunk #25
at hudson.model.RunMap.put(RunMap.java:189)
at jenkins.model.lazy.LazyBuildMixIn.newBuild(LazyBuildMixIn.java:178)
at hudson.model.AbstractProject.newBuild(AbstractProject.java:1019)
at hudson.model.AbstractProject.createExecutable(AbstractProject.java:1209)
at hudson.model.AbstractProject.createExecutable(AbstractProject.java:145)
at hudson.model.Executor$1.call(Executor.java:362)
at hudson.model.Executor$1.call(Executor.java:344)
at hudson.model.Queue._withLock(Queue.java:1345)
at hudson.model.Queue.withLock(Queue.java:1210)
at hudson.model.Executor.run(Executor.java:344)