Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-66506

Ever increasing number of threads in the metrics plugin

    XMLWordPrintable

Details

    Description

      During performance testing, we see that the number of thread that always increases, when it should not as the number of concurrent user is stable (= 1) and the job queue is also a fixed number.

      Looking at thread dump, hundred of thread are:

      "QueueSubTaskMetrics [#6]" Id=350 Group=main WAITING on hudson.model.queue.FutureImpl@6f7dbdac"QueueSubTaskMetrics [#6]" Id=350 Group=main WAITING on hudson.model.queue.FutureImpl@6f7dbdac at java.lang.Object.wait(Native Method) -  waiting on hudson.model.queue.FutureImpl@6f7dbdac at java.lang.Object.wait(Object.java:502) at hudson.remoting.AsyncFutureImpl.get(AsyncFutureImpl.java:79) at jenkins.metrics.impl.JenkinsMetricProviderImpl.lambda$asSupplier$3(JenkinsMetricProviderImpl.java:1142) at jenkins.metrics.impl.JenkinsMetricProviderImpl$$Lambda$670/282561367.get(Unknown Source) at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Number of locked synchronizers = 1 - java.util.concurrent.ThreadPoolExecutor$Worker@22c49544

      an unbounded thread pool is suspected here: https://github.com/jenkinsci/metrics-plugin/blob/21e83be64f85d343c3c9b0e0b0956021d74ade95/src/main/java/jenkins/metrics/impl/JenkinsMetricProviderImpl.java#L836-L839

      jstack and top -H output see attachments.

       

      Attachments

        1. jstack2.txt
          3.41 MB
        2. top-dash-h.txt
          277 kB

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. JENKINS-69817 Sub Task Time in Queue is not getting captured in metrics plugin

          • Major - Major loss of function.
          • Resolved
          depends on

          Improvement - An improvement or enhancement to an existing feature or task. JENKINS-66947 Make ExecutorListener an extension point

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. JENKINS-66941 o.e.j.server LowOnThreads recurring error, Service restart is needed

          • Major - Major loss of function.
          • Reopened
          links to

          Web Link metrics-plugin PR-127

          Activity

            People

              jglick Jesse Glick
              megathaum megathaum
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: