Our Jenkins instance are getting locked up every day.
It seems like this is due to the durable task plugin.
Usins JConsole and connecting to the running java process I find dadlocks and gets this stacktrace:
Name: Computer.threadPoolForRemoting [#179]
State: BLOCKED on hudson.slaves.RetentionStrategy$Demand@1455cecd owned by: jenkins.util.Timer [#1]
Total blocked: 26 Total waited: 522
Stack trace:
hudson.slaves.RetentionStrategy$Demand.check(RetentionStrategy.java:212)
hudson.slaves.RetentionStrategy$Demand.check(RetentionStrategy.java:172)
hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:661)
hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120)
hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180)
- locked java.lang.Object@68ed76f9
jenkins.model.Jenkins.updateComputerList(Jenkins.java:1218)
jenkins.model.Jenkins.setNodes(Jenkins.java:1714)
jenkins.model.Jenkins.removeNode(Jenkins.java:1709)
- locked hudson.model.Hudson@794217b7
hudson.slaves.AbstractCloudSlave.terminate(AbstractCloudSlave.java:65)
org.jenkinsci.plugins.durabletask.executors.OnceRetentionStrategy$1.run(OnceRetentionStrategy.java:125)
- locked hudson.model.Queue@5a25192e
jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
java.util.concurrent.FutureTask.run(FutureTask.java:166)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
java.lang.Thread.run(Thread.java:722)
We are running the Jenkins LTS version 1.596.1 and Durable Task Plugin 1.4.
We also had this problem with Durable Task plugin 1.3.
Running Durable Task plugin 1.2 on Jenkins LTS 1.580.3 seemd to work OK.