Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-51377

High Thread contention around WorkflowRun

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      Builds and Queue maintenance are slowed down because operations require a lock on WorkflowRun to obtain the execution due to lazy-loading, and that lock can be contended heavily as a result. Normally the lock is only briefly held, but the save operation can hold it for a longer period.

      On normal systems the impact should be fairly small but with highly concurrent Pipelines it may result in many threads blocked due to usages like the below (and other within-step operations):

      java.lang.Thread.State: BLOCKED (on object monitor)
      	at org.jenkinsci.plugins.workflow.job.WorkflowRun.getExecution(WorkflowRun.java:844)
      	- waiting to lock <0x00000004ef316030> (a org.jenkinsci.plugins.workflow.job.WorkflowRun)
      	at org.jenkinsci.plugins.workflow.job.WorkflowRun$Owner.get(WorkflowRun.java:1100)
      	at org.jenkinsci.plugins.workflow.cps.CpsStepContext.getExecution(CpsStepContext.java:213)
      	at org.jenkinsci.plugins.workflow.cps.CpsStepContext.getExecution(CpsStepContext.java:95)
      	at org.jenkinsci.plugins.workflow.support.DefaultStepContext.get(DefaultStepContext.java:89)
      	at org.jenkinsci.plugins.workflow.support.steps.ExecutorStepExecution$PlaceholderTask.run(ExecutorStepExecution.java:409)
      	at org.jenkinsci.plugins.workflow.support.steps.ExecutorStepExecution$PlaceholderTask.runForDisplay(ExecutorStepExecution.java:418)
      
      at hudson.util.XStream2.toXMLUTF8(XStream2.java:310)
      	at org.jenkinsci.plugins.workflow.support.PipelineIOUtils.writeByXStream(PipelineIOUtils.java:34)
      	at org.jenkinsci.plugins.workflow.job.WorkflowRun.save(WorkflowRun.java:1256)
      	- locked <0x00000004ef316030> (a org.jenkinsci.plugins.workflow.job.WorkflowRun)
      	at hudson.BulkChange.commit(BulkChange.java:98)
      	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.notifyListeners(CpsFlowExecution.java:1447)
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$3.run(CpsThreadGroup.java:417)
      

        Attachments

          Activity

          svanoort Sam Van Oort created issue -
          svanoort Sam Van Oort made changes -
          Field Original Value New Value
          Status Open [ 1 ] In Progress [ 3 ]
          svanoort Sam Van Oort made changes -
          Status In Progress [ 3 ] Open [ 1 ]
          rsandell rsandell made changes -
          Assignee rsandell [ rsandell ]
          svanoort Sam Van Oort made changes -
          Issue Type Improvement [ 4 ] Bug [ 1 ]
          svanoort Sam Van Oort made changes -
          Summary Improve thread contention around WorkflowRun High Thread contention around WorkflowRun
          svanoort Sam Van Oort made changes -
          Labels performance
          svanoort Sam Van Oort made changes -
          Labels performance performance regression
          cloudbees CloudBees Inc. made changes -
          Remote Link This issue links to "CloudBees Internal CD-555 (Web Link)" [ 20704 ]
          svanoort Sam Van Oort made changes -
          Assignee rsandell [ rsandell ] Sam Van Oort [ svanoort ]
          svanoort Sam Van Oort made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          svanoort Sam Van Oort made changes -
          Status In Progress [ 3 ] In Review [ 10005 ]
          svanoort Sam Van Oort made changes -
          Remote Link This issue links to "workflow-job PR #97 (Web Link)" [ 20836 ]
          petehayes Peter Hayes made changes -
          Attachment image-2018-06-18-09-38-33-398.png [ 42906 ]
          svanoort Sam Van Oort made changes -
          Resolution Done [ 10000 ]
          Status In Review [ 10005 ] Closed [ 6 ]

            People

            Assignee:
            svanoort Sam Van Oort
            Reporter:
            svanoort Sam Van Oort
            Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: