Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-63585

using jenkins version 2.235.2 for CI/CD and its not closing threads automatically

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Component/s: pipeline
    • Labels:
      None
    • Similar Issues:

      Description

      using jenkins version 2.235.2 for CI/CD and its not closing threads automatically.when new pipeline run  its creating new threads  and keeping all  plunging file details in that thread open(around 20000 files on each thread). thread count is getting increase until not restart  Jenkins. when we restart at  that time thread count around 250 and after  4-5 day if we are checking  thread count is around 1000.

      even  we tried with  garbage collection  but after that also thread  count in not decreasing

       

      please  find the  attachment for monitoring  pluging output

        Attachments

          Activity

          Hide
          markewaite Mark Waite added a comment -

          Can you provide more details so that someone else can duplicate the problem you're reporting?

          I run a Jenkins instance with 30 or more agents, several thousand jobs, and intentional stress tests running many bug check jobs did not exceed 600 threads running at a time

          Show
          markewaite Mark Waite added a comment - Can you provide more details so that someone else can duplicate the problem you're reporting? I run a Jenkins instance with 30 or more agents, several thousand jobs, and intentional stress tests running many bug check jobs did not exceed 600 threads running at a time
          Hide
          markewaite Mark Waite added a comment - - edited

          I suspect that there is something in your pipeline shared library that is different than the pipeline shared libraries that I use in my environment.

          Most issues won't get attention from maintainers until there is a precise set of steps that will show the problem. If you'd like more attention on this issue, you'll need to provide step by step instructions that will allow someone else to see the problem in their environment.

          Show
          markewaite Mark Waite added a comment - - edited I suspect that there is something in your pipeline shared library that is different than the pipeline shared libraries that I use in my environment. Most issues won't get attention from maintainers until there is a precise set of steps that will show the problem. If you'd like more attention on this issue, you'll need to provide step by step instructions that will allow someone else to see the problem in their environment.
          Hide
          itramakant ramakant chandrakar added a comment -

          Thanks Mark for you response 

          we are  using around: 10-15  agent and its running around  75 jobs ,we are also using grrovy shared library for ci pipeline. threads of master   is keeping increasing  not get  down till  restart the jenkins.  please  find below high level threads and all threds details in attached pdf  JavaMelody_nodes_9_17_20.pdf

          Show
          itramakant ramakant chandrakar added a comment - Thanks Mark for you response  we are  using around: 10-15  agent and its running around  75 jobs ,we are also using grrovy shared library for ci pipeline. threads of master   is keeping increasing  not get  down till  restart the jenkins.  please  find below high level threads and all threds details in attached pdf   JavaMelody_nodes_9_17_20.pdf
          Hide
          itramakant ramakant chandrakar added a comment -

          please find attached  thread dumps also thredd dumps.txt

          Show
          itramakant ramakant chandrakar added a comment - please find attached  thread dumps also  thredd dumps.txt
          Hide
          itramakant ramakant chandrakar added a comment -

          *on 18 sep- 2020 master agent has *
          Threads on masterjenkins: Number = 626, Maximum = 669, Total started = 24,437 Details
          *on 21 sep- 2020 master agent has *
          Threads on masterjenkins: Number = 819, Maximum = 842, Total started = 47,800 Details

          Show
          itramakant ramakant chandrakar added a comment - *on 18 sep- 2020 master agent has * Threads on masterjenkins: Number = 626, Maximum = 669, Total started = 24,437 Details *on 21 sep- 2020 master agent has * Threads on masterjenkins: Number = 819, Maximum = 842, Total started = 47,800 Details
          Hide
          itramakant ramakant chandrakar added a comment -

          all waiting threads has below details and we are using jdk 11 is there any issue with jdk 11

          "Thread-4214" daemon prio=5 WAITING
          java.base@11.0.4/java.lang.Object.wait(Native Method)
          java.base@11.0.4/java.lang.Object.wait(Object.java:328)
          java.naming@11.0.4/com.sun.jndi.ldap.Connection.pauseReader(Connection.java:790)
          java.naming@11.0.4/com.sun.jndi.ldap.Connection.run(Connection.java:930)
          java.base@11.0.4/java.lang.Thread.run(Thread.java:834)

          "Thread-4219" daemon prio=5 WAITING
          java.base@11.0.4/java.lang.Object.wait(Native Method)
          java.base@11.0.4/java.lang.Object.wait(Object.java:328)
          java.naming@11.0.4/com.sun.jndi.ldap.Connection.pauseReader(Connection.java:790)
          java.naming@11.0.4/com.sun.jndi.ldap.Connection.run(Connection.java:930)
          java.base@11.0.4/java.lang.Thread.run(Thread.java:834)

          Show
          itramakant ramakant chandrakar added a comment - all waiting threads has below details and we are using jdk 11 is there any issue with jdk 11 "Thread-4214" daemon prio=5 WAITING java.base@11.0.4/java.lang.Object.wait(Native Method) java.base@11.0.4/java.lang.Object.wait(Object.java:328) java.naming@11.0.4/com.sun.jndi.ldap.Connection.pauseReader(Connection.java:790) java.naming@11.0.4/com.sun.jndi.ldap.Connection.run(Connection.java:930) java.base@11.0.4/java.lang.Thread.run(Thread.java:834) "Thread-4219" daemon prio=5 WAITING java.base@11.0.4/java.lang.Object.wait(Native Method) java.base@11.0.4/java.lang.Object.wait(Object.java:328) java.naming@11.0.4/com.sun.jndi.ldap.Connection.pauseReader(Connection.java:790) java.naming@11.0.4/com.sun.jndi.ldap.Connection.run(Connection.java:930) java.base@11.0.4/java.lang.Thread.run(Thread.java:834)
          Hide
          markewaite Mark Waite added a comment -

          I'm not aware of any issue that is specific to Java 11 and threads in Jenkins. There are issues that are specific to Java 11, but I'm not aware of any Java 11 issues that are related to threading. My first guess is that you would be likely to see the same problem on Java 8 as you are seeing on Java 11. The biggest challenge is to identify what is different in your environment compared to other environments.

          Show
          markewaite Mark Waite added a comment - I'm not aware of any issue that is specific to Java 11 and threads in Jenkins. There are issues that are specific to Java 11, but I'm not aware of any Java 11 issues that are related to threading. My first guess is that you would be likely to see the same problem on Java 8 as you are seeing on Java 11. The biggest challenge is to identify what is different in your environment compared to other environments.

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            itramakant ramakant chandrakar
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated: