Stuck pod causing Jenkins break

This issue is archived. You can view it, but you can't modify it. Learn more

XMLWordPrintable

      Jenkins agent is removed from jenkins somehow, but agent pod is still running on kubernetes cluster, so k8s pod tries to re-connect every 10s.

      Jenkins is refusing connection:

      2025-11-04 09:52:00.882+0000 [id=2333]    INFO    h.TcpSlaveAgentListener$ConnectionHandler#run: Connection #1099 from /10.123.2.253:29485 failed: null
      2025-11-04 09:52:00.883+0000 [id=2334]    INFO    h.TcpSlaveAgentListener$ConnectionHandler#run: Accepted JNLP4-connect connection #1100 from /10.123.2.253:6644
      2025-11-04 09:52:00.890+0000 [id=2287]    INFO    o.j.r.p.i.ConnectionHeadersFilterLayer#onRecv: [JNLP4-connect connection from core.prod.publishdrive.com/10.123.2.253:6644] Refusing headers from remote: Unknown client name: livesales-43-storytel-1591-wntv6-0q38w 

      Connection id is increasing and several threads are open in Java:

      Handling GET /tcpSlaveAgentListener/ from 10.123.2.253 : Jetty (winstone)-26 

      Something seems to be leaking, because after a while, I can see 100% CPU usage on Garbage Collector threads:

      1991964 mainten+  20   0 6099204   2.7g  10604 R  92.4  34.5 215:20.83 GC Thread#1
      1991966 mainten+  20   0 6099204   2.7g  10604 R  88.4  34.5 215:26.16 GC Thread#3
      1991311 mainten+  20   0 6099204   2.7g  10604 R  82.7  34.5 215:21.00 GC Thread#0
      1991965 mainten+  20   0 6099204   2.7g  10604 R  82.7  34.5 215:13.79 GC Thread#2 

      I sent kill -3, here are the stack trace: java.txt

      Can you please:

      • fix memory leak
      • help to set connection retry limit in kubernetes agent pod

      ?

      Thanks in advice!

       

      BR,

      Tamas

       

       

            Assignee:
            Unassigned
            Reporter:
            Tamas
            Archiver:
            Jenkins Service Account

              Created:
              Updated:
              Archived: