Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-27476

Plugin casue deadlock on Jenkins LTS 1.596.1

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Critical
    • Resolution: Duplicate
    • durable-task-plugin
    • None
    • RHEL 6.x
      Jenkins LTS 1.596.1
      Durable Task Plugin 1.4

    Description

      Our Jenkins instance are getting locked up every day.
      It seems like this is due to the durable task plugin.

      Usins JConsole and connecting to the running java process I find dadlocks and gets this stacktrace:

      Name: Computer.threadPoolForRemoting [#179]
State: BLOCKED on hudson.slaves.RetentionStrategy$Demand@1455cecd owned by: jenkins.util.Timer [#1]
Total blocked: 26  Total waited: 522

Stack trace: 
hudson.slaves.RetentionStrategy$Demand.check(RetentionStrategy.java:212)
hudson.slaves.RetentionStrategy$Demand.check(RetentionStrategy.java:172)
hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:661)
hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120)
hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180)
   - locked java.lang.Object@68ed76f9
jenkins.model.Jenkins.updateComputerList(Jenkins.java:1218)
jenkins.model.Jenkins.setNodes(Jenkins.java:1714)
jenkins.model.Jenkins.removeNode(Jenkins.java:1709)
   - locked hudson.model.Hudson@794217b7
hudson.slaves.AbstractCloudSlave.terminate(AbstractCloudSlave.java:65)
org.jenkinsci.plugins.durabletask.executors.OnceRetentionStrategy$1.run(OnceRetentionStrategy.java:125)
   - locked hudson.model.Queue@5a25192e
jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
java.util.concurrent.FutureTask.run(FutureTask.java:166)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
java.lang.Thread.run(Thread.java:722)

      We are running the Jenkins LTS version 1.596.1 and Durable Task Plugin 1.4.
      We also had this problem with Durable Task plugin 1.3.

      Running Durable Task plugin 1.2 on Jenkins LTS 1.580.3 seemd to work OK.

      Attachments

        Issue Links

          depends on

          Bug - A problem which impairs or prevents the functions of the product. JENKINS-27565 Nodes can be removed as idle before the assigned tasks have started

          • Major - Major loss of function.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            View 10 older comments
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Stephen Connolly
            Path:
            changelog.html
            http://jenkins-ci.org/commit/jenkins/46dc6850edb1d7ef52592794b15e69db7dfbed1a
            Log:
            Noting merges

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Stephen Connolly Path: changelog.html http://jenkins-ci.org/commit/jenkins/46dc6850edb1d7ef52592794b15e69db7dfbed1a Log: Noting merges JENKINS-15355 JENKINS-21618 JENKINS-22941 JENKINS-25938 JENKINS-26391 JENKINS-26900 JENKINS-27476 JENKINS-27563 JENKINS-27564 JENKINS-27565 JENKINS-27566 Fixing link text for JENKINS-6167
            jglick Jesse Glick added a comment -

            This is filed in a plugin and so by definition cannot be lts-candidate. stephenconnolly what is its status?

            jglick Jesse Glick added a comment - This is filed in a plugin and so by definition cannot be lts-candidate . stephenconnolly what is its status?
            danielbeck Daniel Beck added a comment -

            pablaasmo Does this issue still occur in Jenkins 1.607 or higher, or can it be considered resolved?

            danielbeck Daniel Beck added a comment - pablaasmo Does this issue still occur in Jenkins 1.607 or higher, or can it be considered resolved?
            jglick Jesse Glick added a comment -

            Closing as covered by the core fix unless I hear information to the contrary.

            jglick Jesse Glick added a comment - Closing as covered by the core fix unless I hear information to the contrary.
            pablaasmo Per Arnold Blaasmo added a comment -

            I have not seen this issue again. So I think it is ok to close

            pablaasmo Per Arnold Blaasmo added a comment - I have not seen this issue again. So I think it is ok to close

            People

              stephenconnolly Stephen Connolly
              pablaasmo Per Arnold Blaasmo
              Votes:
              2 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: