Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-66446

SEVERE: Uncaught exception in Engine thread Thread[Thread-0,5,main]

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      I'm using jenkins helm chart for my jenkins controller (master) with persistence EFS volume together with Kubernetes plugin with inbound-agent (jdk11) pods for my agents (slaves) over websocket connection

      It seems that the Agent (slave) pods are unable to re-connect the controller (master) after a reboot to the controller (master).

      It seems that the process exit with exit code 0 from some reason instead of reconnecting.

       

      jenkins controller (master) log

      2021-08-24 13:26:33.976+0000 [id=64]    INFO    o.c.j.p.k.KubernetesLauncher#launch: Agent has already been launched, activating: jenkins-agent-3q3j5
      

       

      jenkins agent (slave) log:

      Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1
      Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1SEVERE: Uncaught exception in Engine thread Thread[Thread-0,5,main]
      java.lang.NoClassDefFoundError: jenkins/slaves/restarter/JnlpSlaveRestarterInstaller 
          at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$FindEffectiveRestarters$1.onReconnect(JnlpSlaveRestarterInstaller.java:91) 
          at hudson.remoting.EngineListenerSplitter.onReconnect(EngineListenerSplitter.java:54) 
          at hudson.remoting.Engine.runWebSocket(Engine.java:687) 
          at hudson.remoting.Engine.run(Engine.java:496)
      Caused by: java.lang.ClassNotFoundException: jenkins.slaves.restarter.JnlpSlaveRestarterInstaller     
          at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471) 
          at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:215)   at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589) 
          at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) ... 4 more
      

       

      jenkins agent (slave) pod status:

      NAME                              READY   STATUS     RESTARTS   AGE
      jenkins-agent-3q3j5               1/2     NotReady   0          15m
      
      
          lastState: {}
          name: jnlp
          ready: false
          restartCount: 0
          started: false
          state:
            terminated:
              containerID: docker://9b490eafd5078fa95cbd915bed16bcee767c1005a7a96158a2017d1551fba87b
              exitCode: 0
              finishedAt: "2021-09-08T15:00:07Z"
              reason: Completed
              startedAt: "2021-09-08T14:53:19Z"
      
      

       

       inbound-agent conf

          - name: JENKINS_SECRET
            value: 271983729187392173921798379281739821793b
          - name: JENKINS_AGENT_NAME
            value: jenkins-agent-3q3j5
          - name: DOCKER_HOST
            value: tcp://localhost:2375
          - name: JENKINS_WEB_SOCKET
            value: "true"
          - name: JAVA_OPTS
            value: -Xms512m -Xmx1500m
          - name: JENKINS_NAME
            value: jenkins-agent-3q3j5
          - name: JENKINS_AGENT_WORKDIR
            value: /home/jenkins/agent
          - name: JENKINS_URL
            value: http://jenkins:8080/
          - name: AWS_DEFAULT_REGION
            value: us-east-1
          - name: AWS_REGION
            value: us-east-1
          - name: AWS_ROLE_ARN
            value: arn:aws:iam::123456789:role/jenkins-agent-pod
          - name: AWS_WEB_IDENTITY_TOKEN_FILE
            value: /var/run/secrets/eks.amazonaws.com/serviceaccount/token
      

       

      jenkins agent (slave) pod process

      /opt/java/openjdk/bin/java -Xms512m -Xmx1500m -cp /usr/share/jenkins/agent.jar hudson.remoting.jnlp.Main -headless -url http://jenkins:8080/ -workDir /home /jenkins/agent -webSocket 271983729187392173921798379281739821793b jenkins-agent-3q3j5
      

       

      same behavior in

      jenkins/jenkins:2.303.1-jdk11
      jenkins/inbound-agent:4.10-2-jdk11
      

       

      It seems that this issue happens also on windows agent (slave)

       

      might be related: https://issues.jenkins.io/browse/JENKINS-59910

        Attachments

          Activity

          dordor dor s created issue -
          dordor dor s made changes -
          Field Original Value New Value
          Description Agent (slaves) pods can't re-connect to master after a reboot

          jenkins controller (master) log
          {code:java}
          2021-08-24 13:26:33.976+0000 [id=64] INFO o.c.j.p.k.KubernetesLauncher#launch: Agent has already been launched, activating: jenkins-agent-3q3j5
          {code}
          jenkins agent (slave) log:
          {code:java}
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1SEVERE: Uncaught exception in Engine thread Thread[Thread-0,5,main]java.lang.NoClassDefFoundError: jenkins/slaves/restarter/JnlpSlaveRestarterInstaller at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$FindEffectiveRestarters$1.onReconnect(JnlpSlaveRestarterInstaller.java:91) at hudson.remoting.EngineListenerSplitter.onReconnect(EngineListenerSplitter.java:54) at hudson.remoting.Engine.runWebSocket(Engine.java:687) at hudson.remoting.Engine.run(Engine.java:496)Caused by: java.lang.ClassNotFoundException: jenkins.slaves.restarter.JnlpSlaveRestarterInstaller at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471) at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:215) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) ... 4 more
          {code}
           
          Agent (slaves) pods can't re-connect to master after a reboot

          jenkins controller (master) log
          {code:java}
          2021-08-24 13:26:33.976+0000 [id=64] INFO o.c.j.p.k.KubernetesLauncher#launch: Agent has already been launched, activating: jenkins-agent-3q3j5
          {code}
          jenkins agent (slave) log:
          {code:java}
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1SEVERE: Uncaught exception in Engine thread Thread[Thread-0,5,main]java.lang.NoClassDefFoundError: jenkins/slaves/restarter/JnlpSlaveRestarterInstaller at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$FindEffectiveRestarters$1.onReconnect(JnlpSlaveRestarterInstaller.java:91) at hudson.remoting.EngineListenerSplitter.onReconnect(EngineListenerSplitter.java:54) at hudson.remoting.Engine.runWebSocket(Engine.java:687) at hudson.remoting.Engine.run(Engine.java:496)Caused by: java.lang.ClassNotFoundException: jenkins.slaves.restarter.JnlpSlaveRestarterInstaller at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471) at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:215) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) ... 4 more
          {code}
           

          might be related: https://issues.jenkins.io/browse/JENKINS-59910
          dordor dor s made changes -
          Description Agent (slaves) pods can't re-connect to master after a reboot

          jenkins controller (master) log
          {code:java}
          2021-08-24 13:26:33.976+0000 [id=64] INFO o.c.j.p.k.KubernetesLauncher#launch: Agent has already been launched, activating: jenkins-agent-3q3j5
          {code}
          jenkins agent (slave) log:
          {code:java}
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1SEVERE: Uncaught exception in Engine thread Thread[Thread-0,5,main]java.lang.NoClassDefFoundError: jenkins/slaves/restarter/JnlpSlaveRestarterInstaller at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$FindEffectiveRestarters$1.onReconnect(JnlpSlaveRestarterInstaller.java:91) at hudson.remoting.EngineListenerSplitter.onReconnect(EngineListenerSplitter.java:54) at hudson.remoting.Engine.runWebSocket(Engine.java:687) at hudson.remoting.Engine.run(Engine.java:496)Caused by: java.lang.ClassNotFoundException: jenkins.slaves.restarter.JnlpSlaveRestarterInstaller at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471) at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:215) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) ... 4 more
          {code}
           

          might be related: https://issues.jenkins.io/browse/JENKINS-59910
          I'm using jenkins helm [chart|https://github.com/jenkinsci/helm-charts/tree/main/charts/jenkins] for my jenkins controller (master) with persistence EFS volume together with Kubernetes plugin with 

          Agent (slaves) pods can't re-connect to master after a reboot

          jenkins controller (master) log
          {code:java}
          2021-08-24 13:26:33.976+0000 [id=64] INFO o.c.j.p.k.KubernetesLauncher#launch: Agent has already been launched, activating: jenkins-agent-3q3j5
          {code}
          jenkins agent (slave) log:
          {code:java}
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1SEVERE: Uncaught exception in Engine thread Thread[Thread-0,5,main]
          java.lang.NoClassDefFoundError: jenkins/slaves/restarter/JnlpSlaveRestarterInstaller
              at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$FindEffectiveRestarters$1.onReconnect(JnlpSlaveRestarterInstaller.java:91)
              at hudson.remoting.EngineListenerSplitter.onReconnect(EngineListenerSplitter.java:54)
              at hudson.remoting.Engine.runWebSocket(Engine.java:687)
              at hudson.remoting.Engine.run(Engine.java:496)
          Caused by: java.lang.ClassNotFoundException: jenkins.slaves.restarter.JnlpSlaveRestarterInstaller
              at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
              at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:215) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
              at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) ... 4 more
          {code}
           

          might be related: https://issues.jenkins.io/browse/JENKINS-59910
          dordor dor s made changes -
          Description I'm using jenkins helm [chart|https://github.com/jenkinsci/helm-charts/tree/main/charts/jenkins] for my jenkins controller (master) with persistence EFS volume together with Kubernetes plugin with 

          Agent (slaves) pods can't re-connect to master after a reboot

          jenkins controller (master) log
          {code:java}
          2021-08-24 13:26:33.976+0000 [id=64] INFO o.c.j.p.k.KubernetesLauncher#launch: Agent has already been launched, activating: jenkins-agent-3q3j5
          {code}
          jenkins agent (slave) log:
          {code:java}
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1SEVERE: Uncaught exception in Engine thread Thread[Thread-0,5,main]
          java.lang.NoClassDefFoundError: jenkins/slaves/restarter/JnlpSlaveRestarterInstaller
              at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$FindEffectiveRestarters$1.onReconnect(JnlpSlaveRestarterInstaller.java:91)
              at hudson.remoting.EngineListenerSplitter.onReconnect(EngineListenerSplitter.java:54)
              at hudson.remoting.Engine.runWebSocket(Engine.java:687)
              at hudson.remoting.Engine.run(Engine.java:496)
          Caused by: java.lang.ClassNotFoundException: jenkins.slaves.restarter.JnlpSlaveRestarterInstaller
              at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
              at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:215) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
              at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) ... 4 more
          {code}
           

          might be related: https://issues.jenkins.io/browse/JENKINS-59910
          I'm using jenkins helm [chart|https://github.com/jenkinsci/helm-charts/tree/main/charts/jenkins] for my jenkins controller (master) with persistence EFS volume together with Kubernetes plugin with inbound-agent pods for my agents (slaves)

          It seems that the Agent (slave) pods are unable to re-connect the controller (master) after a reboot to the controller (master), like after a master upgrade.

          jenkins controller (master) log
          {code:java}
          2021-08-24 13:26:33.976+0000 [id=64] INFO o.c.j.p.k.KubernetesLauncher#launch: Agent has already been launched, activating: jenkins-agent-3q3j5
          {code}
          jenkins agent (slave) log:
          {code:java}
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1SEVERE: Uncaught exception in Engine thread Thread[Thread-0,5,main]
          java.lang.NoClassDefFoundError: jenkins/slaves/restarter/JnlpSlaveRestarterInstaller
              at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$FindEffectiveRestarters$1.onReconnect(JnlpSlaveRestarterInstaller.java:91)
              at hudson.remoting.EngineListenerSplitter.onReconnect(EngineListenerSplitter.java:54)
              at hudson.remoting.Engine.runWebSocket(Engine.java:687)
              at hudson.remoting.Engine.run(Engine.java:496)
          Caused by: java.lang.ClassNotFoundException: jenkins.slaves.restarter.JnlpSlaveRestarterInstaller
              at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
              at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:215) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
              at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) ... 4 more
          {code}
           inbound-agent conf

           
          {code:java}
              - name: JENKINS_SECRET
                value: 271983729187392173921798379281739821793b
              - name: JENKINS_AGENT_NAME
                value: jenkins-agent-3q3j5
              - name: DOCKER_HOST
                value: tcp://localhost:2375
              - name: JENKINS_WEB_SOCKET
                value: "true"
              - name: JAVA_OPTS
                value: -Xms512m -Xmx1500m
              - name: JENKINS_NAME
                value: jenkins-agent-3q3j5
              - name: JENKINS_AGENT_WORKDIR
                value: /home/jenkins/agent
              - name: JENKINS_URL
                value: http://jenkins:8080/
              - name: AWS_DEFAULT_REGION
                value: us-east-1
              - name: AWS_REGION
                value: us-east-1
              - name: AWS_ROLE_ARN
                value: arn:aws:iam::123456789:role/jenkins-agent-pod
              - name: AWS_WEB_IDENTITY_TOKEN_FILE
                value: /var/run/secrets/eks.amazonaws.com/serviceaccount/token
          {code}
           

          might be related: https://issues.jenkins.io/browse/JENKINS-59910
          dordor dor s made changes -
          Description I'm using jenkins helm [chart|https://github.com/jenkinsci/helm-charts/tree/main/charts/jenkins] for my jenkins controller (master) with persistence EFS volume together with Kubernetes plugin with inbound-agent pods for my agents (slaves)

          It seems that the Agent (slave) pods are unable to re-connect the controller (master) after a reboot to the controller (master), like after a master upgrade.

          jenkins controller (master) log
          {code:java}
          2021-08-24 13:26:33.976+0000 [id=64] INFO o.c.j.p.k.KubernetesLauncher#launch: Agent has already been launched, activating: jenkins-agent-3q3j5
          {code}
          jenkins agent (slave) log:
          {code:java}
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1SEVERE: Uncaught exception in Engine thread Thread[Thread-0,5,main]
          java.lang.NoClassDefFoundError: jenkins/slaves/restarter/JnlpSlaveRestarterInstaller
              at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$FindEffectiveRestarters$1.onReconnect(JnlpSlaveRestarterInstaller.java:91)
              at hudson.remoting.EngineListenerSplitter.onReconnect(EngineListenerSplitter.java:54)
              at hudson.remoting.Engine.runWebSocket(Engine.java:687)
              at hudson.remoting.Engine.run(Engine.java:496)
          Caused by: java.lang.ClassNotFoundException: jenkins.slaves.restarter.JnlpSlaveRestarterInstaller
              at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
              at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:215) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
              at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) ... 4 more
          {code}
           inbound-agent conf

           
          {code:java}
              - name: JENKINS_SECRET
                value: 271983729187392173921798379281739821793b
              - name: JENKINS_AGENT_NAME
                value: jenkins-agent-3q3j5
              - name: DOCKER_HOST
                value: tcp://localhost:2375
              - name: JENKINS_WEB_SOCKET
                value: "true"
              - name: JAVA_OPTS
                value: -Xms512m -Xmx1500m
              - name: JENKINS_NAME
                value: jenkins-agent-3q3j5
              - name: JENKINS_AGENT_WORKDIR
                value: /home/jenkins/agent
              - name: JENKINS_URL
                value: http://jenkins:8080/
              - name: AWS_DEFAULT_REGION
                value: us-east-1
              - name: AWS_REGION
                value: us-east-1
              - name: AWS_ROLE_ARN
                value: arn:aws:iam::123456789:role/jenkins-agent-pod
              - name: AWS_WEB_IDENTITY_TOKEN_FILE
                value: /var/run/secrets/eks.amazonaws.com/serviceaccount/token
          {code}
           

          might be related: https://issues.jenkins.io/browse/JENKINS-59910
          I'm using jenkins helm [chart|https://github.com/jenkinsci/helm-charts/tree/main/charts/jenkins] for my jenkins controller (master) with persistence EFS volume together with Kubernetes plugin with inbound-agent pods for my agents (slaves) over websocket connection

          It seems that the Agent (slave) pods are unable to re-connect the controller (master) after a reboot to the controller (master), like after a master upgrade.

           

          jenkins controller (master) log
          {code:java}
          2021-08-24 13:26:33.976+0000 [id=64] INFO o.c.j.p.k.KubernetesLauncher#launch: Agent has already been launched, activating: jenkins-agent-3q3j5
          {code}
           

          jenkins agent (slave) log:
          {code:java}
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1SEVERE: Uncaught exception in Engine thread Thread[Thread-0,5,main]
          java.lang.NoClassDefFoundError: jenkins/slaves/restarter/JnlpSlaveRestarterInstaller
              at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$FindEffectiveRestarters$1.onReconnect(JnlpSlaveRestarterInstaller.java:91)
              at hudson.remoting.EngineListenerSplitter.onReconnect(EngineListenerSplitter.java:54)
              at hudson.remoting.Engine.runWebSocket(Engine.java:687)
              at hudson.remoting.Engine.run(Engine.java:496)
          Caused by: java.lang.ClassNotFoundException: jenkins.slaves.restarter.JnlpSlaveRestarterInstaller
              at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
              at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:215) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
              at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) ... 4 more
          {code}
           

           inbound-agent conf
          {code:java}
              - name: JENKINS_SECRET
                value: 271983729187392173921798379281739821793b
              - name: JENKINS_AGENT_NAME
                value: jenkins-agent-3q3j5
              - name: DOCKER_HOST
                value: tcp://localhost:2375
              - name: JENKINS_WEB_SOCKET
                value: "true"
              - name: JAVA_OPTS
                value: -Xms512m -Xmx1500m
              - name: JENKINS_NAME
                value: jenkins-agent-3q3j5
              - name: JENKINS_AGENT_WORKDIR
                value: /home/jenkins/agent
              - name: JENKINS_URL
                value: http://jenkins:8080/
              - name: AWS_DEFAULT_REGION
                value: us-east-1
              - name: AWS_REGION
                value: us-east-1
              - name: AWS_ROLE_ARN
                value: arn:aws:iam::123456789:role/jenkins-agent-pod
              - name: AWS_WEB_IDENTITY_TOKEN_FILE
                value: /var/run/secrets/eks.amazonaws.com/serviceaccount/token
          {code}
           

          might be related: https://issues.jenkins.io/browse/JENKINS-59910
          dordor dor s made changes -
          Priority Major [ 3 ] Critical [ 2 ]
          dordor dor s made changes -
          Description I'm using jenkins helm [chart|https://github.com/jenkinsci/helm-charts/tree/main/charts/jenkins] for my jenkins controller (master) with persistence EFS volume together with Kubernetes plugin with inbound-agent pods for my agents (slaves) over websocket connection

          It seems that the Agent (slave) pods are unable to re-connect the controller (master) after a reboot to the controller (master), like after a master upgrade.

           

          jenkins controller (master) log
          {code:java}
          2021-08-24 13:26:33.976+0000 [id=64] INFO o.c.j.p.k.KubernetesLauncher#launch: Agent has already been launched, activating: jenkins-agent-3q3j5
          {code}
           

          jenkins agent (slave) log:
          {code:java}
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1SEVERE: Uncaught exception in Engine thread Thread[Thread-0,5,main]
          java.lang.NoClassDefFoundError: jenkins/slaves/restarter/JnlpSlaveRestarterInstaller
              at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$FindEffectiveRestarters$1.onReconnect(JnlpSlaveRestarterInstaller.java:91)
              at hudson.remoting.EngineListenerSplitter.onReconnect(EngineListenerSplitter.java:54)
              at hudson.remoting.Engine.runWebSocket(Engine.java:687)
              at hudson.remoting.Engine.run(Engine.java:496)
          Caused by: java.lang.ClassNotFoundException: jenkins.slaves.restarter.JnlpSlaveRestarterInstaller
              at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
              at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:215) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
              at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) ... 4 more
          {code}
           

           inbound-agent conf
          {code:java}
              - name: JENKINS_SECRET
                value: 271983729187392173921798379281739821793b
              - name: JENKINS_AGENT_NAME
                value: jenkins-agent-3q3j5
              - name: DOCKER_HOST
                value: tcp://localhost:2375
              - name: JENKINS_WEB_SOCKET
                value: "true"
              - name: JAVA_OPTS
                value: -Xms512m -Xmx1500m
              - name: JENKINS_NAME
                value: jenkins-agent-3q3j5
              - name: JENKINS_AGENT_WORKDIR
                value: /home/jenkins/agent
              - name: JENKINS_URL
                value: http://jenkins:8080/
              - name: AWS_DEFAULT_REGION
                value: us-east-1
              - name: AWS_REGION
                value: us-east-1
              - name: AWS_ROLE_ARN
                value: arn:aws:iam::123456789:role/jenkins-agent-pod
              - name: AWS_WEB_IDENTITY_TOKEN_FILE
                value: /var/run/secrets/eks.amazonaws.com/serviceaccount/token
          {code}
           

          might be related: https://issues.jenkins.io/browse/JENKINS-59910
          I'm using jenkins *helm* [chart|https://github.com/jenkinsci/helm-charts/tree/main/charts/jenkins] for my jenkins controller (master) with persistence *EFS* volume together with *Kubernetes* plugin with inbound-agent pods for my agents (slaves) over *websocket* connection

          It seems that the Agent (slave) pods are unable to re-connect the controller (master) after a reboot to the controller (master), like after a master upgrade.

           

          jenkins controller (master) log
          {code:java}
          2021-08-24 13:26:33.976+0000 [id=64] INFO o.c.j.p.k.KubernetesLauncher#launch: Agent has already been launched, activating: jenkins-agent-3q3j5
          {code}
           

          jenkins agent (slave) log:
          {code:java}
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1SEVERE: Uncaught exception in Engine thread Thread[Thread-0,5,main]
          java.lang.NoClassDefFoundError: jenkins/slaves/restarter/JnlpSlaveRestarterInstaller
              at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$FindEffectiveRestarters$1.onReconnect(JnlpSlaveRestarterInstaller.java:91)
              at hudson.remoting.EngineListenerSplitter.onReconnect(EngineListenerSplitter.java:54)
              at hudson.remoting.Engine.runWebSocket(Engine.java:687)
              at hudson.remoting.Engine.run(Engine.java:496)
          Caused by: java.lang.ClassNotFoundException: jenkins.slaves.restarter.JnlpSlaveRestarterInstaller
              at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
              at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:215) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
              at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) ... 4 more
          {code}
           

           inbound-agent conf
          {code:java}
              - name: JENKINS_SECRET
                value: 271983729187392173921798379281739821793b
              - name: JENKINS_AGENT_NAME
                value: jenkins-agent-3q3j5
              - name: DOCKER_HOST
                value: tcp://localhost:2375
              - name: JENKINS_WEB_SOCKET
                value: "true"
              - name: JAVA_OPTS
                value: -Xms512m -Xmx1500m
              - name: JENKINS_NAME
                value: jenkins-agent-3q3j5
              - name: JENKINS_AGENT_WORKDIR
                value: /home/jenkins/agent
              - name: JENKINS_URL
                value: http://jenkins:8080/
              - name: AWS_DEFAULT_REGION
                value: us-east-1
              - name: AWS_REGION
                value: us-east-1
              - name: AWS_ROLE_ARN
                value: arn:aws:iam::123456789:role/jenkins-agent-pod
              - name: AWS_WEB_IDENTITY_TOKEN_FILE
                value: /var/run/secrets/eks.amazonaws.com/serviceaccount/token
          {code}
           

          jenkins agent (slave) *pod* process
          {code:java}
          /opt/java/openjdk/bin/java -Xms512m -Xmx1500m -cp /usr/share/jenkins/agent.jar hudson.remoting.jnlp.Main -headless -url http://jenkins:8080/ -workDir /home /jenkins/agent -webSocket c5dffe1f23fdsfb43f44545dfa167e2323sdfd6 jenkins-agent-k1ay3
          {code}
           

          might be related: https://issues.jenkins.io/browse/JENKINS-59910
          dordor dor s made changes -
          Description I'm using jenkins *helm* [chart|https://github.com/jenkinsci/helm-charts/tree/main/charts/jenkins] for my jenkins controller (master) with persistence *EFS* volume together with *Kubernetes* plugin with inbound-agent pods for my agents (slaves) over *websocket* connection

          It seems that the Agent (slave) pods are unable to re-connect the controller (master) after a reboot to the controller (master), like after a master upgrade.

           

          jenkins controller (master) log
          {code:java}
          2021-08-24 13:26:33.976+0000 [id=64] INFO o.c.j.p.k.KubernetesLauncher#launch: Agent has already been launched, activating: jenkins-agent-3q3j5
          {code}
           

          jenkins agent (slave) log:
          {code:java}
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1SEVERE: Uncaught exception in Engine thread Thread[Thread-0,5,main]
          java.lang.NoClassDefFoundError: jenkins/slaves/restarter/JnlpSlaveRestarterInstaller
              at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$FindEffectiveRestarters$1.onReconnect(JnlpSlaveRestarterInstaller.java:91)
              at hudson.remoting.EngineListenerSplitter.onReconnect(EngineListenerSplitter.java:54)
              at hudson.remoting.Engine.runWebSocket(Engine.java:687)
              at hudson.remoting.Engine.run(Engine.java:496)
          Caused by: java.lang.ClassNotFoundException: jenkins.slaves.restarter.JnlpSlaveRestarterInstaller
              at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
              at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:215) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
              at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) ... 4 more
          {code}
           

           inbound-agent conf
          {code:java}
              - name: JENKINS_SECRET
                value: 271983729187392173921798379281739821793b
              - name: JENKINS_AGENT_NAME
                value: jenkins-agent-3q3j5
              - name: DOCKER_HOST
                value: tcp://localhost:2375
              - name: JENKINS_WEB_SOCKET
                value: "true"
              - name: JAVA_OPTS
                value: -Xms512m -Xmx1500m
              - name: JENKINS_NAME
                value: jenkins-agent-3q3j5
              - name: JENKINS_AGENT_WORKDIR
                value: /home/jenkins/agent
              - name: JENKINS_URL
                value: http://jenkins:8080/
              - name: AWS_DEFAULT_REGION
                value: us-east-1
              - name: AWS_REGION
                value: us-east-1
              - name: AWS_ROLE_ARN
                value: arn:aws:iam::123456789:role/jenkins-agent-pod
              - name: AWS_WEB_IDENTITY_TOKEN_FILE
                value: /var/run/secrets/eks.amazonaws.com/serviceaccount/token
          {code}
           

          jenkins agent (slave) *pod* process
          {code:java}
          /opt/java/openjdk/bin/java -Xms512m -Xmx1500m -cp /usr/share/jenkins/agent.jar hudson.remoting.jnlp.Main -headless -url http://jenkins:8080/ -workDir /home /jenkins/agent -webSocket c5dffe1f23fdsfb43f44545dfa167e2323sdfd6 jenkins-agent-k1ay3
          {code}
           

          might be related: https://issues.jenkins.io/browse/JENKINS-59910
          I'm using jenkins *helm* [chart|https://github.com/jenkinsci/helm-charts/tree/main/charts/jenkins] for my jenkins controller (master) with persistence *EFS* volume together with *Kubernetes* plugin with inbound-agent pods for my agents (slaves) over *websocket* connection

          It seems that the Agent (slave) pods are unable to re-connect the controller (master) after a reboot to the controller (master), like after a master upgrade.

           

          jenkins controller (master) log
          {code:java}
          2021-08-24 13:26:33.976+0000 [id=64] INFO o.c.j.p.k.KubernetesLauncher#launch: Agent has already been launched, activating: jenkins-agent-3q3j5
          {code}
           

          jenkins agent (slave) log:
          {code:java}
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1SEVERE: Uncaught exception in Engine thread Thread[Thread-0,5,main]
          java.lang.NoClassDefFoundError: jenkins/slaves/restarter/JnlpSlaveRestarterInstaller
              at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$FindEffectiveRestarters$1.onReconnect(JnlpSlaveRestarterInstaller.java:91)
              at hudson.remoting.EngineListenerSplitter.onReconnect(EngineListenerSplitter.java:54)
              at hudson.remoting.Engine.runWebSocket(Engine.java:687)
              at hudson.remoting.Engine.run(Engine.java:496)
          Caused by: java.lang.ClassNotFoundException: jenkins.slaves.restarter.JnlpSlaveRestarterInstaller
              at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
              at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:215) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
              at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) ... 4 more
          {code}
           

           inbound-agent conf
          {code:java}
              - name: JENKINS_SECRET
                value: 271983729187392173921798379281739821793b
              - name: JENKINS_AGENT_NAME
                value: jenkins-agent-3q3j5
              - name: DOCKER_HOST
                value: tcp://localhost:2375
              - name: JENKINS_WEB_SOCKET
                value: "true"
              - name: JAVA_OPTS
                value: -Xms512m -Xmx1500m
              - name: JENKINS_NAME
                value: jenkins-agent-3q3j5
              - name: JENKINS_AGENT_WORKDIR
                value: /home/jenkins/agent
              - name: JENKINS_URL
                value: http://jenkins:8080/
              - name: AWS_DEFAULT_REGION
                value: us-east-1
              - name: AWS_REGION
                value: us-east-1
              - name: AWS_ROLE_ARN
                value: arn:aws:iam::123456789:role/jenkins-agent-pod
              - name: AWS_WEB_IDENTITY_TOKEN_FILE
                value: /var/run/secrets/eks.amazonaws.com/serviceaccount/token
          {code}
           

          jenkins agent (slave) *pod* process
          {code:java}
          /opt/java/openjdk/bin/java -Xms512m -Xmx1500m -cp /usr/share/jenkins/agent.jar hudson.remoting.jnlp.Main -headless -url http://jenkins:8080/ -workDir /home /jenkins/agent -webSocket 271983729187392173921798379281739821793b jenkins-agent-3q3j5
          {code}
           

          might be related: https://issues.jenkins.io/browse/JENKINS-59910
          dordor dor s made changes -
          Description I'm using jenkins *helm* [chart|https://github.com/jenkinsci/helm-charts/tree/main/charts/jenkins] for my jenkins controller (master) with persistence *EFS* volume together with *Kubernetes* plugin with inbound-agent pods for my agents (slaves) over *websocket* connection

          It seems that the Agent (slave) pods are unable to re-connect the controller (master) after a reboot to the controller (master), like after a master upgrade.

           

          jenkins controller (master) log
          {code:java}
          2021-08-24 13:26:33.976+0000 [id=64] INFO o.c.j.p.k.KubernetesLauncher#launch: Agent has already been launched, activating: jenkins-agent-3q3j5
          {code}
           

          jenkins agent (slave) log:
          {code:java}
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1SEVERE: Uncaught exception in Engine thread Thread[Thread-0,5,main]
          java.lang.NoClassDefFoundError: jenkins/slaves/restarter/JnlpSlaveRestarterInstaller
              at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$FindEffectiveRestarters$1.onReconnect(JnlpSlaveRestarterInstaller.java:91)
              at hudson.remoting.EngineListenerSplitter.onReconnect(EngineListenerSplitter.java:54)
              at hudson.remoting.Engine.runWebSocket(Engine.java:687)
              at hudson.remoting.Engine.run(Engine.java:496)
          Caused by: java.lang.ClassNotFoundException: jenkins.slaves.restarter.JnlpSlaveRestarterInstaller
              at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
              at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:215) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
              at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) ... 4 more
          {code}
           

           inbound-agent conf
          {code:java}
              - name: JENKINS_SECRET
                value: 271983729187392173921798379281739821793b
              - name: JENKINS_AGENT_NAME
                value: jenkins-agent-3q3j5
              - name: DOCKER_HOST
                value: tcp://localhost:2375
              - name: JENKINS_WEB_SOCKET
                value: "true"
              - name: JAVA_OPTS
                value: -Xms512m -Xmx1500m
              - name: JENKINS_NAME
                value: jenkins-agent-3q3j5
              - name: JENKINS_AGENT_WORKDIR
                value: /home/jenkins/agent
              - name: JENKINS_URL
                value: http://jenkins:8080/
              - name: AWS_DEFAULT_REGION
                value: us-east-1
              - name: AWS_REGION
                value: us-east-1
              - name: AWS_ROLE_ARN
                value: arn:aws:iam::123456789:role/jenkins-agent-pod
              - name: AWS_WEB_IDENTITY_TOKEN_FILE
                value: /var/run/secrets/eks.amazonaws.com/serviceaccount/token
          {code}
           

          jenkins agent (slave) *pod* process
          {code:java}
          /opt/java/openjdk/bin/java -Xms512m -Xmx1500m -cp /usr/share/jenkins/agent.jar hudson.remoting.jnlp.Main -headless -url http://jenkins:8080/ -workDir /home /jenkins/agent -webSocket 271983729187392173921798379281739821793b jenkins-agent-3q3j5
          {code}
           

          might be related: https://issues.jenkins.io/browse/JENKINS-59910
          I'm using jenkins *helm* [chart|https://github.com/jenkinsci/helm-charts/tree/main/charts/jenkins] for my jenkins controller (master) with persistence *EFS* volume together with *Kubernetes* plugin with inbound-agent pods for my agents (slaves) over *websocket* connection

          It seems that the Agent (slave) pods are unable to re-connect the controller (master) after a reboot to the controller (master), like after a master upgrade.

           

          jenkins controller (master) log
          {code:java}
          2021-08-24 13:26:33.976+0000 [id=64] INFO o.c.j.p.k.KubernetesLauncher#launch: Agent has already been launched, activating: jenkins-agent-3q3j5
          {code}
           

          jenkins agent (slave) log:
          {code:java}
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1SEVERE: Uncaught exception in Engine thread Thread[Thread-0,5,main]
          java.lang.NoClassDefFoundError: jenkins/slaves/restarter/JnlpSlaveRestarterInstaller
              at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$FindEffectiveRestarters$1.onReconnect(JnlpSlaveRestarterInstaller.java:91)
              at hudson.remoting.EngineListenerSplitter.onReconnect(EngineListenerSplitter.java:54)
              at hudson.remoting.Engine.runWebSocket(Engine.java:687)
              at hudson.remoting.Engine.run(Engine.java:496)
          Caused by: java.lang.ClassNotFoundException: jenkins.slaves.restarter.JnlpSlaveRestarterInstaller
              at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
              at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:215) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
              at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) ... 4 more
          {code}
           

           inbound-agent conf
          {code:java}
              - name: JENKINS_SECRET
                value: 271983729187392173921798379281739821793b
              - name: JENKINS_AGENT_NAME
                value: jenkins-agent-3q3j5
              - name: DOCKER_HOST
                value: tcp://localhost:2375
              - name: JENKINS_WEB_SOCKET
                value: "true"
              - name: JAVA_OPTS
                value: -Xms512m -Xmx1500m
              - name: JENKINS_NAME
                value: jenkins-agent-3q3j5
              - name: JENKINS_AGENT_WORKDIR
                value: /home/jenkins/agent
              - name: JENKINS_URL
                value: http://jenkins:8080/
              - name: AWS_DEFAULT_REGION
                value: us-east-1
              - name: AWS_REGION
                value: us-east-1
              - name: AWS_ROLE_ARN
                value: arn:aws:iam::123456789:role/jenkins-agent-pod
              - name: AWS_WEB_IDENTITY_TOKEN_FILE
                value: /var/run/secrets/eks.amazonaws.com/serviceaccount/token
          {code}
           

          jenkins agent (slave) *pod* process
          {code:java}
          /opt/java/openjdk/bin/java -Xms512m -Xmx1500m -cp /usr/share/jenkins/agent.jar hudson.remoting.jnlp.Main -headless -url http://jenkins:8080/ -workDir /home /jenkins/agent -webSocket 271983729187392173921798379281739821793b jenkins-agent-3q3j5
          {code}
           

          It seems that this issue happens also on *windows* agent (slave)

           

          might be related: https://issues.jenkins.io/browse/JENKINS-59910
          dordor dor s made changes -
          Description I'm using jenkins *helm* [chart|https://github.com/jenkinsci/helm-charts/tree/main/charts/jenkins] for my jenkins controller (master) with persistence *EFS* volume together with *Kubernetes* plugin with inbound-agent pods for my agents (slaves) over *websocket* connection

          It seems that the Agent (slave) pods are unable to re-connect the controller (master) after a reboot to the controller (master), like after a master upgrade.

           

          jenkins controller (master) log
          {code:java}
          2021-08-24 13:26:33.976+0000 [id=64] INFO o.c.j.p.k.KubernetesLauncher#launch: Agent has already been launched, activating: jenkins-agent-3q3j5
          {code}
           

          jenkins agent (slave) log:
          {code:java}
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1SEVERE: Uncaught exception in Engine thread Thread[Thread-0,5,main]
          java.lang.NoClassDefFoundError: jenkins/slaves/restarter/JnlpSlaveRestarterInstaller
              at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$FindEffectiveRestarters$1.onReconnect(JnlpSlaveRestarterInstaller.java:91)
              at hudson.remoting.EngineListenerSplitter.onReconnect(EngineListenerSplitter.java:54)
              at hudson.remoting.Engine.runWebSocket(Engine.java:687)
              at hudson.remoting.Engine.run(Engine.java:496)
          Caused by: java.lang.ClassNotFoundException: jenkins.slaves.restarter.JnlpSlaveRestarterInstaller
              at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
              at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:215) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
              at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) ... 4 more
          {code}
           

           inbound-agent conf
          {code:java}
              - name: JENKINS_SECRET
                value: 271983729187392173921798379281739821793b
              - name: JENKINS_AGENT_NAME
                value: jenkins-agent-3q3j5
              - name: DOCKER_HOST
                value: tcp://localhost:2375
              - name: JENKINS_WEB_SOCKET
                value: "true"
              - name: JAVA_OPTS
                value: -Xms512m -Xmx1500m
              - name: JENKINS_NAME
                value: jenkins-agent-3q3j5
              - name: JENKINS_AGENT_WORKDIR
                value: /home/jenkins/agent
              - name: JENKINS_URL
                value: http://jenkins:8080/
              - name: AWS_DEFAULT_REGION
                value: us-east-1
              - name: AWS_REGION
                value: us-east-1
              - name: AWS_ROLE_ARN
                value: arn:aws:iam::123456789:role/jenkins-agent-pod
              - name: AWS_WEB_IDENTITY_TOKEN_FILE
                value: /var/run/secrets/eks.amazonaws.com/serviceaccount/token
          {code}
           

          jenkins agent (slave) *pod* process
          {code:java}
          /opt/java/openjdk/bin/java -Xms512m -Xmx1500m -cp /usr/share/jenkins/agent.jar hudson.remoting.jnlp.Main -headless -url http://jenkins:8080/ -workDir /home /jenkins/agent -webSocket 271983729187392173921798379281739821793b jenkins-agent-3q3j5
          {code}
           

          It seems that this issue happens also on *windows* agent (slave)

           

          might be related: https://issues.jenkins.io/browse/JENKINS-59910
          I'm using jenkins *helm* [chart|https://github.com/jenkinsci/helm-charts/tree/main/charts/jenkins] for my jenkins controller (master) with persistence *EFS* volume together with *Kubernetes* plugin with inbound-agent pods for my agents (slaves) over *websocket* connection

          It seems that the Agent (slave) pods are unable to re-connect the controller (master) after a reboot to the controller (master).

          It seems that the process exit with *exit code 0* from some reason instead of reconnecting.

           

          jenkins controller (master) log
          {code:java}
          2021-08-24 13:26:33.976+0000 [id=64] INFO o.c.j.p.k.KubernetesLauncher#launch: Agent has already been launched, activating: jenkins-agent-3q3j5
          {code}
           

          jenkins agent (slave) log:
          {code:java}
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1SEVERE: Uncaught exception in Engine thread Thread[Thread-0,5,main]
          java.lang.NoClassDefFoundError: jenkins/slaves/restarter/JnlpSlaveRestarterInstaller
              at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$FindEffectiveRestarters$1.onReconnect(JnlpSlaveRestarterInstaller.java:91)
              at hudson.remoting.EngineListenerSplitter.onReconnect(EngineListenerSplitter.java:54)
              at hudson.remoting.Engine.runWebSocket(Engine.java:687)
              at hudson.remoting.Engine.run(Engine.java:496)
          Caused by: java.lang.ClassNotFoundException: jenkins.slaves.restarter.JnlpSlaveRestarterInstaller
              at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
              at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:215) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
              at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) ... 4 more
          {code}
           

          jenkins agent (slave) pod status:
          {code:java}
          NAME READY STATUS RESTARTS AGE
          jenkins-agent-3q3j5 1/2 NotReady 0 15m


              lastState: {}
              name: jnlp
              ready: false
              restartCount: 0
              started: false
              state:
                terminated:
                  containerID: docker://9b490eafd5078fa95cbd915bed16bcee767c1005a7a96158a2017d1551fba87b
                  exitCode: 0
                  finishedAt: "2021-09-08T15:00:07Z"
                  reason: Completed
                  startedAt: "2021-09-08T14:53:19Z"

          {code}
           

           inbound-agent conf
          {code:java}
              - name: JENKINS_SECRET
                value: 271983729187392173921798379281739821793b
              - name: JENKINS_AGENT_NAME
                value: jenkins-agent-3q3j5
              - name: DOCKER_HOST
                value: tcp://localhost:2375
              - name: JENKINS_WEB_SOCKET
                value: "true"
              - name: JAVA_OPTS
                value: -Xms512m -Xmx1500m
              - name: JENKINS_NAME
                value: jenkins-agent-3q3j5
              - name: JENKINS_AGENT_WORKDIR
                value: /home/jenkins/agent
              - name: JENKINS_URL
                value: http://jenkins:8080/
              - name: AWS_DEFAULT_REGION
                value: us-east-1
              - name: AWS_REGION
                value: us-east-1
              - name: AWS_ROLE_ARN
                value: arn:aws:iam::123456789:role/jenkins-agent-pod
              - name: AWS_WEB_IDENTITY_TOKEN_FILE
                value: /var/run/secrets/eks.amazonaws.com/serviceaccount/token
          {code}
           

          jenkins agent (slave) *pod* process
          {code:java}
          /opt/java/openjdk/bin/java -Xms512m -Xmx1500m -cp /usr/share/jenkins/agent.jar hudson.remoting.jnlp.Main -headless -url http://jenkins:8080/ -workDir /home /jenkins/agent -webSocket 271983729187392173921798379281739821793b jenkins-agent-3q3j5
          {code}
           

          It seems that this issue happens also on *windows* agent (slave)

           

          might be related: https://issues.jenkins.io/browse/JENKINS-59910
          dordor dor s made changes -
          Description I'm using jenkins *helm* [chart|https://github.com/jenkinsci/helm-charts/tree/main/charts/jenkins] for my jenkins controller (master) with persistence *EFS* volume together with *Kubernetes* plugin with inbound-agent pods for my agents (slaves) over *websocket* connection

          It seems that the Agent (slave) pods are unable to re-connect the controller (master) after a reboot to the controller (master).

          It seems that the process exit with *exit code 0* from some reason instead of reconnecting.

           

          jenkins controller (master) log
          {code:java}
          2021-08-24 13:26:33.976+0000 [id=64] INFO o.c.j.p.k.KubernetesLauncher#launch: Agent has already been launched, activating: jenkins-agent-3q3j5
          {code}
           

          jenkins agent (slave) log:
          {code:java}
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1SEVERE: Uncaught exception in Engine thread Thread[Thread-0,5,main]
          java.lang.NoClassDefFoundError: jenkins/slaves/restarter/JnlpSlaveRestarterInstaller
              at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$FindEffectiveRestarters$1.onReconnect(JnlpSlaveRestarterInstaller.java:91)
              at hudson.remoting.EngineListenerSplitter.onReconnect(EngineListenerSplitter.java:54)
              at hudson.remoting.Engine.runWebSocket(Engine.java:687)
              at hudson.remoting.Engine.run(Engine.java:496)
          Caused by: java.lang.ClassNotFoundException: jenkins.slaves.restarter.JnlpSlaveRestarterInstaller
              at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
              at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:215) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
              at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) ... 4 more
          {code}
           

          jenkins agent (slave) pod status:
          {code:java}
          NAME READY STATUS RESTARTS AGE
          jenkins-agent-3q3j5 1/2 NotReady 0 15m


              lastState: {}
              name: jnlp
              ready: false
              restartCount: 0
              started: false
              state:
                terminated:
                  containerID: docker://9b490eafd5078fa95cbd915bed16bcee767c1005a7a96158a2017d1551fba87b
                  exitCode: 0
                  finishedAt: "2021-09-08T15:00:07Z"
                  reason: Completed
                  startedAt: "2021-09-08T14:53:19Z"

          {code}
           

           inbound-agent conf
          {code:java}
              - name: JENKINS_SECRET
                value: 271983729187392173921798379281739821793b
              - name: JENKINS_AGENT_NAME
                value: jenkins-agent-3q3j5
              - name: DOCKER_HOST
                value: tcp://localhost:2375
              - name: JENKINS_WEB_SOCKET
                value: "true"
              - name: JAVA_OPTS
                value: -Xms512m -Xmx1500m
              - name: JENKINS_NAME
                value: jenkins-agent-3q3j5
              - name: JENKINS_AGENT_WORKDIR
                value: /home/jenkins/agent
              - name: JENKINS_URL
                value: http://jenkins:8080/
              - name: AWS_DEFAULT_REGION
                value: us-east-1
              - name: AWS_REGION
                value: us-east-1
              - name: AWS_ROLE_ARN
                value: arn:aws:iam::123456789:role/jenkins-agent-pod
              - name: AWS_WEB_IDENTITY_TOKEN_FILE
                value: /var/run/secrets/eks.amazonaws.com/serviceaccount/token
          {code}
           

          jenkins agent (slave) *pod* process
          {code:java}
          /opt/java/openjdk/bin/java -Xms512m -Xmx1500m -cp /usr/share/jenkins/agent.jar hudson.remoting.jnlp.Main -headless -url http://jenkins:8080/ -workDir /home /jenkins/agent -webSocket 271983729187392173921798379281739821793b jenkins-agent-3q3j5
          {code}
           

          It seems that this issue happens also on *windows* agent (slave)

           

          might be related: https://issues.jenkins.io/browse/JENKINS-59910
          I'm using jenkins *helm* [chart|https://github.com/jenkinsci/helm-charts/tree/main/charts/jenkins] for my jenkins controller (master) with persistence *EFS* volume together with *Kubernetes* plugin with *inbound-agent* (jdk11) pods for my agents (slaves) over *websocket* connection

          It seems that the Agent (slave) pods are unable to re-connect the controller (master) after a reboot to the controller (master).

          It seems that the process exit with *exit code 0* from some reason instead of reconnecting.

           

          jenkins controller (master) log
          {code:java}
          2021-08-24 13:26:33.976+0000 [id=64] INFO o.c.j.p.k.KubernetesLauncher#launch: Agent has already been launched, activating: jenkins-agent-3q3j5
          {code}
           

          jenkins agent (slave) log:
          {code:java}
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1
          Aug 24, 2021 1:23:56 PM hudson.remoting.Engine lambda$new$1SEVERE: Uncaught exception in Engine thread Thread[Thread-0,5,main]
          java.lang.NoClassDefFoundError: jenkins/slaves/restarter/JnlpSlaveRestarterInstaller
              at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$FindEffectiveRestarters$1.onReconnect(JnlpSlaveRestarterInstaller.java:91)
              at hudson.remoting.EngineListenerSplitter.onReconnect(EngineListenerSplitter.java:54)
              at hudson.remoting.Engine.runWebSocket(Engine.java:687)
              at hudson.remoting.Engine.run(Engine.java:496)
          Caused by: java.lang.ClassNotFoundException: jenkins.slaves.restarter.JnlpSlaveRestarterInstaller
              at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
              at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:215) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
              at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) ... 4 more
          {code}
           

          jenkins agent (slave) pod status:
          {code:java}
          NAME READY STATUS RESTARTS AGE
          jenkins-agent-3q3j5 1/2 NotReady 0 15m


              lastState: {}
              name: jnlp
              ready: false
              restartCount: 0
              started: false
              state:
                terminated:
                  containerID: docker://9b490eafd5078fa95cbd915bed16bcee767c1005a7a96158a2017d1551fba87b
                  exitCode: 0
                  finishedAt: "2021-09-08T15:00:07Z"
                  reason: Completed
                  startedAt: "2021-09-08T14:53:19Z"

          {code}
           

           inbound-agent conf
          {code:java}
              - name: JENKINS_SECRET
                value: 271983729187392173921798379281739821793b
              - name: JENKINS_AGENT_NAME
                value: jenkins-agent-3q3j5
              - name: DOCKER_HOST
                value: tcp://localhost:2375
              - name: JENKINS_WEB_SOCKET
                value: "true"
              - name: JAVA_OPTS
                value: -Xms512m -Xmx1500m
              - name: JENKINS_NAME
                value: jenkins-agent-3q3j5
              - name: JENKINS_AGENT_WORKDIR
                value: /home/jenkins/agent
              - name: JENKINS_URL
                value: http://jenkins:8080/
              - name: AWS_DEFAULT_REGION
                value: us-east-1
              - name: AWS_REGION
                value: us-east-1
              - name: AWS_ROLE_ARN
                value: arn:aws:iam::123456789:role/jenkins-agent-pod
              - name: AWS_WEB_IDENTITY_TOKEN_FILE
                value: /var/run/secrets/eks.amazonaws.com/serviceaccount/token
          {code}
           

          jenkins agent (slave) *pod* process
          {code:java}
          /opt/java/openjdk/bin/java -Xms512m -Xmx1500m -cp /usr/share/jenkins/agent.jar hudson.remoting.jnlp.Main -headless -url http://jenkins:8080/ -workDir /home /jenkins/agent -webSocket 271983729187392173921798379281739821793b jenkins-agent-3q3j5
          {code}
           

          same behavior in
          {code:java}
          jenkins/jenkins:2.303.1-jdk11
          jenkins/inbound-agent:4.10-2-jdk11
          {code}
           

          It seems that this issue happens also on *windows* agent (slave)

           

          might be related: https://issues.jenkins.io/browse/JENKINS-59910
          dordor dor s made changes -
          Environment jenkins k8s controller (master) pod version: jenkins/jenkins:2.289.3-jdk11
          jenkins k8s agent (slave) pod version:  jenkins/inbound-agent:4.9-1-jdk11
          Kubernetes plugin: 1.30.1
          Jenkins: 2.303.1
          jenkins k8s controller (master) pod version: jenkins/jenkins:2.289.3-jdk11
          jenkins k8s agent (slave) pod version:  jenkins/inbound-agent:4.9-1-jdk11
          Kubernetes plugin: 1.30.1
          dordor dor s made changes -
          Labels kubernetes remoting java11-compatibility kubernetes remoting
          bganicky Bohdan Ganicky made changes -
          Comment [ Having the same issue with following environment:
          [Jenkins 2.303.3:|https://www.jenkins.io/]
           - JDK11
           - in docker version 20.10.7, build 20.10.7-0ubuntu5~21.04.2
           - single node container
           - Kubernetes plugin 1.30.10
           - OS: Ubuntu 21.04 (GNU/Linux 5.11.0-40-generic x86_64) in VMware ESXi 7


          K8S for running the workload:
           - 3x beefy underutilized machines
           - Rancher 2.6
           - Kubernetes 1.21.5
           - docker 20.10.9-ce
           - OS: OpenSuSE Leap 15.3, kernel 5.3.18-59.34-default), bare-metal

          Workload is Jenkins multibranch pipeline or local pipeline (both behaves similar in terms of symptoms):

          FIO in bash running aginst NFS share is mounted inside container or as volume (both behaves similar in terms of symptoms)

          {noformat}
          pipeline {
              agent {
                  kubernetes {
                      yaml '''
          apiVersion: v1
          kind: Pod
          metadata:
           name: jnks-fio-inside-mount-pod
           labels:
             app: jnks-fio-inside-mount
          spec:
           containers:
            - name: jnlp
              image: 'jenkins/inbound-agent:4.11-1-jdk11'
              args: ['\$(JENKINS_SECRET)', '\$(JENKINS_NAME)']
            - name: jnks-fio-inside-mount-container
              image: opensuse-leap-st:1.0
              command:
               - sleep
              args:
               - infinity
              securityContext:
               privileged: true
          '''
                      // Can also wrap individual steps:
                      // container('shell') {
                      // sh 'hostname'
                      // }
                      defaultContainer 'jnks-fio-inside-mount-container'
                  }
              }
              stages {
                  stage ('Mount share') {
                      steps{
                          sh 'mount -vvv -o vers=4.1 -o nconnect=16 server:/fio-jnks-opensuse-inside-1 /mnt'
                      }
                  }
                  stage ('Run FIO') {
                      steps{
                          sh 'fio --verify=crc32c --buffer_compress_percentage=0 --buffer_compress_chunk=0 --size=10G --directory=/mnt --name=fio --ioengine=libaio --fallocate=none --iodepth=2 --rw=write --bs=512k --direct=1 --numjobs=8 --nrfiles=8 --runtime=600 --group_reporting'
                      }
                  }
                  stage('Delete files') {
                      steps {
                          sh 'rm -rf /mnt/fio*'
                      }
                  }
                      
                  }
          triggers { upstream(upstreamProjects: "${env.JOB_NAME}", threshold: hudson.model.Result.SUCCESS) }
          }

          {noformat}

          Symptoms:
           - Workload is running like 50 times without issue and then jnlp container end up in Not ready state. Log for jnlp container shows following:
          {noformat}
          Warning: SECRET is defined twice in command-line arguments and the environment variable
          Warning: AGENT_NAME is defined twice in command-line arguments and the environment variable
          Nov 24, 2021 3:15:38 PM hudson.remoting.jnlp.Main createEngine
          INFO: Setting up agent: patamat-k8s-nfsv4-1-fio-inside-mount-opensuse-83-016q0-fl-67xhz
          Nov 24, 2021 3:15:38 PM hudson.remoting.jnlp.Main$CuiListener <init>
          INFO: Jenkins agent is running in headless mode.
          Nov 24, 2021 3:15:38 PM hudson.remoting.Engine startEngine
          INFO: Using Remoting version: 4.11
          Nov 24, 2021 3:15:38 PM org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir
          INFO: Using /home/jenkins/agent/remoting as a remoting work directory
          Nov 24, 2021 3:15:38 PM org.jenkinsci.remoting.engine.WorkDirManager setupLogging
          INFO: Both error and output logs will be printed to /home/jenkins/agent/remoting
          Nov 24, 2021 3:15:38 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: WebSocket connection open
          Nov 24, 2021 3:15:38 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Connected
          Nov 24, 2021 3:18:15 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Write side closed
          Nov 24, 2021 3:18:15 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Read side closed
          Nov 24, 2021 3:18:15 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Terminated
          Nov 24, 2021 3:18:15 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Read side closed
          Nov 24, 2021 3:18:15 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Read side closed
          Nov 24, 2021 3:18:15 PM hudson.remoting.Engine lambda$new$1
          SEVERE: Uncaught exception in Engine thread Thread[Thread-0,5,main]
          java.lang.NoClassDefFoundError: jenkins/slaves/restarter/JnlpSlaveRestarterInstaller
          at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$FindEffectiveRestarters$1.onReconnect(JnlpSlaveRestarterInstaller.java:91)
          at hudson.remoting.EngineListenerSplitter.onReconnect(EngineListenerSplitter.java:54)
          at hudson.remoting.Engine.runWebSocket(Engine.java:687)
          at hudson.remoting.Engine.run(Engine.java:496)
          Caused by: java.lang.ClassNotFoundException: jenkins.slaves.restarter.JnlpSlaveRestarterInstaller
          at java.base/java.net.URLClassLoader.findClass(Unknown Source)
          at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:215)
          at java.base/java.lang.ClassLoader.loadClass(Unknown Source)
          at java.base/java.lang.ClassLoader.loadClass(Unknown Source)
          ... 4 more
          {noformat}

          Pipeline log in Jenkins shows following:
          {noformat}
          Timed out waiting for websocket connection. You should increase the value of system property org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator.websocketConnectionTimeout currently set at 30 seconds
          {noformat}

          OR

          {noformat}
          java.net.ProtocolException: Expected HTTP 101 response but was '500 Internal Server Error'

          at okhttp3.internal.ws.RealWebSocket.checkResponse(RealWebSocket.java:229)

          at okhttp3.internal.ws.RealWebSocket$2.onResponse(RealWebSocket.java:196)

          at okhttp3.RealCall$AsyncCall.execute(RealCall.java:203)

          at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)

          at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)

          at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)

          at java.base/java.lang.Thread.run(Thread.java:829)

          ERROR: Process exited immediately after creation. See output below

          Executing sh script inside container jnks-fio-inside-mount-container of pod patamat-k8s-nfsv4-1-fio-inside-mount-opensuse-46-ptr5k-qz-fv3zk



          Process exited immediately after creation. Check logs above for more details.
          {noformat}
          ]
          bganicky Bohdan Ganicky made changes -
          Comment [ Seems similar to https://issues.jenkins.io/browse/JENKINS-67167 as well ]
          bganicky Bohdan Ganicky made changes -
          Comment [ UPDATE: Faulty containers are replaced by working ones suddenly (this was not the case). Stuck containers are left in K8S cluster (in Error or Not ready state for jnlp container) and builds are continuing to work. Is this expected behavior?
          UPDATE2: Faulty containers are not always replaced and job in Jenkins fails in that case ]

            People

            Assignee:
            jthompson Jeff Thompson
            Reporter:
            dordor dor s
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated: