Stuck pod causing Jenkins break

XMLWordPrintable

    • Type: Bug
    • Resolution: Unresolved
    • Priority: Major
    • Component/s: kubernetes-plugin
    • None
    • Environment:
      Official Jenkins Docker image 2.534
      Kubernetes plugin 4384.v1b_6367f393d9

      Jenkins agent is removed from jenkins somehow, but agent pod is still running on kubernetes cluster, so k8s pod tries to re-connect every 10s.

      Jenkins is refusing connection:

      2025-11-04 09:52:00.882+0000 [id=2333]    INFO    h.TcpSlaveAgentListener$ConnectionHandler#run: Connection #1099 from /10.123.2.253:29485 failed: null
      2025-11-04 09:52:00.883+0000 [id=2334]    INFO    h.TcpSlaveAgentListener$ConnectionHandler#run: Accepted JNLP4-connect connection #1100 from /10.123.2.253:6644
      2025-11-04 09:52:00.890+0000 [id=2287]    INFO    o.j.r.p.i.ConnectionHeadersFilterLayer#onRecv: [JNLP4-connect connection from core.prod.publishdrive.com/10.123.2.253:6644] Refusing headers from remote: Unknown client name: livesales-43-storytel-1591-wntv6-0q38w 

      Connection id is increasing and several threads are open in Java:

      Handling GET /tcpSlaveAgentListener/ from 10.123.2.253 : Jetty (winstone)-26 

      Something seems to be leaking, because after a while, I can see 100% CPU usage on Garbage Collector threads:

      1991964 mainten+  20   0 6099204   2.7g  10604 R  92.4  34.5 215:20.83 GC Thread#1
      1991966 mainten+  20   0 6099204   2.7g  10604 R  88.4  34.5 215:26.16 GC Thread#3
      1991311 mainten+  20   0 6099204   2.7g  10604 R  82.7  34.5 215:21.00 GC Thread#0
      1991965 mainten+  20   0 6099204   2.7g  10604 R  82.7  34.5 215:13.79 GC Thread#2 

      I sent kill -3, here are the stack trace: java.txt

      Can you please:

      • fix memory leak
      • help to set connection retry limit in kubernetes agent pod

      ?

      Thanks in advice!

       

      BR,

      Tamas

       

       

            Assignee:
            Unassigned
            Reporter:
            Tamas
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: