Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-63538

New agents are suspended without any stdout or sdterr

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Component/s: core, ec2-plugin
    • Labels:
      None
    • Environment:
      Jenkins 2.235.5
      Amazon Linux 2
      openjdk version "1.8.0_242"
    • Similar Issues:

      Description

      I have Jenkins 2.235.5 which is running under docker in AWS EC2 virtual machine.

      While running build Jenkins provision new agent Linux virtual machine and tries start a agent on it.

      For some reason in 20-90% cases agent could not be started. And I cannot understand what can influence such behavior and what can influence on probability of the issue.

      stdout and stderr - are empty for Agent process.

      I can get only log on Jenkins Master side:

      Complete!
       Aug 27, 2020 1:20:12 PM hudson.plugins.ec2.EC2Cloud
       INFO: Verifying: which scp
       /usr/bin/scp
       Aug 27, 2020 1:20:13 PM hudson.plugins.ec2.EC2Cloud
       INFO: Copying remoting.jar to: /tmp
       Aug 27, 2020 1:20:13 PM hudson.plugins.ec2.EC2Cloud
       INFO: Launching remoting agent (via Trilead SSH2 Connection): java -jar /tmp/remoting.jar -workDir /home/ec2-user
       <===[JENKINS REMOTING CAPACITY]===>Remoting version: 4.3
       This is a Unix agent

       

      If I try to kill agent process on Agent virtual machine then I receive next error on Jenkins Master side:

       

      2020-08-27 13:30:36.112+0000 [id=33046] INFO h.r.SynchronousCommandTransport$ReaderThread#run: I/O error in channel EC2 (agent) - centos (i-0415552e93fd24e69)
       java.io.EOFException
       at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2736)
       at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3211)
       at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:896)
       at java.io.ObjectInputStream.<init>(ObjectInputStream.java:358)
       at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:49)
       at hudson.remoting.Command.readFrom(Command.java:142)
       at hudson.remoting.Command.readFrom(Command.java:128)
       at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:35)
       at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:63)
       Caused: java.io.IOException: Unexpected termination of the channel
       at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:77)
       2020-08-27 13:30:36.123+0000 [id=33052] WARNING hudson.remoting.Request$2#run: Failed to send back a reply to the request hudson.remoting.Request$2@331a2e08
       java.io.EOFException
       at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2736)
       at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3211)
       at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:896)
       at java.io.ObjectInputStream.<init>(ObjectInputStream.java:358)
       at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:49)
       at hudson.remoting.Command.readFrom(Command.java:142)
       at hudson.remoting.Command.readFrom(Command.java:128)
       at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:35)
       at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:63)
       Caused: java.io.IOException: Unexpected termination of the channel
       at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:77)
       Caused: hudson.remoting.ChannelClosedException: Channel "hudson.remoting.Channel@67d34b2d:EC2 (agent) - centos (i-0415552e93fd24e69)": channel is already closed
       at hudson.remoting.Channel.send(Channel.java:760)
       at hudson.remoting.Request$2.run(Request.java:382)
       at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
       at org.jenkinsci.remoting.CallableDecorator.call(CallableDecorator.java:18)
       at hudson.remoting.CallableDecoratorList$1.call(CallableDecoratorList.java:21)
       at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
       at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71)
       at java.util.concurrent.FutureTask.run(FutureTask.java:266)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
       at java.lang.Thread.run(Thread.java:748)

       

      Please help to fix issue or find some workaround.

      Thanks.

       

       

        Attachments

          Activity

          There are no comments yet on this issue.

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            yaraslau_haradzetski Yaraslau Haradzetski
            Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated: