Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-51863

EC2 Plugin: Connectivity issues

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Minor Minor
    • ec2-plugin
    • EC2 Plugin 1.39
      Master
      - Jenkins ver. 2.124
      - Java 1.8.0_171 (64 bit)
      Slave
      - M5.Large
      - Windows Server 2016
      - Java 1.8.0_171 (64 bit)

      About half of the time a new node is spun up, there is a connectivity issue between master and slave. The master successfully connects to the slave, but then the connection fails soon-after. When a true successful connection occurs (about half of the time), which I define as having the `Agent successfully connected and online` message appearing in the log, the connection remains stable and is not an issue.

       

      First time poster - if any additional information is needed, please let me know.

       

      Logged error below:

      FINER: Node ***** is ready 
      ***** booted at 1528718031000 
      Connecting to ***** with WinRM as administrator 
      Connected with WinRM. 
      Creating tmp directory if it does not exist 
      slave.jar sent remotely. Bootstrapping it 
      <===[JENKINS REMOTING CAPACITY]===>Remoting version: 3.20 
      This is a Windows agent 
      Connection terminated 
      ERROR: Failed to install restarter 
      Ouch: 
      hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown: Command Close created at 
      at hudson.remoting.Request.abort(Request.java:340) 
      at hudson.remoting.Channel.terminate(Channel.java:1038) 
      at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1271) 
      at hudson.remoting.Channel$1.handle(Channel.java:565) 
      at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:87) 
      Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to ***** 
      at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741) 
      at hudson.remoting.Request.call(Request.java:202) 
      at hudson.remoting.Channel.call(Channel.java:954) 
      at org.jenkinsci.modules.upstart_slave_installer.SlaveInstallerFactoryImpl.createIfApplicable(SlaveInstallerFactoryImpl.java:29)
      at org.jenkinsci.modules.slave_installer.SlaveInstallerFactory.createIfApplicable(SlaveInstallerFactory.java:29) 
      at org.jenkinsci.modules.slave_installer.SlaveInstallerFactory.createFor(SlaveInstallerFactory.java:46) 
      at org.jenkinsci.modules.slave_installer.impl.ComputerListenerImpl.onOnline(ComputerListenerImpl.java:30) 
      at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:620) 
      at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:416) 
      at hudson.plugins.ec2.win.EC2WindowsLauncher.launch(EC2WindowsLauncher.java:70) 
      at hudson.plugins.ec2.EC2ComputerLauncher.launch(EC2ComputerLauncher.java:122) 
      at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:288) 
      at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) 
      at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71) 
      at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
      at java.lang.Thread.run(Thread.java:748) 
      Caused by: hudson.remoting.Channel$OrderlyShutdown: Command Close created at 
      ... 3 more 
      Caused by: Command Close created at 
      at hudson.remoting.Command.<init>(Command.java:65) 
      at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:1265) 
      at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:1263) 
      at hudson.remoting.Channel.close(Channel.java:1436) 
      at hudson.remoting.Channel.close(Channel.java:1403) 
      at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1270) 
      ... 2 more 
      Command Close created at 
      at hudson.remoting.Command.<init>(Command.java:65) 
      at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:1265) 
      at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:1263) 
      at hudson.remoting.Channel.close(Channel.java:1436) 
      at hudson.remoting.Channel.close(Channel.java:1403) 
      at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1270) 
      Caused: hudson.remoting.Channel$OrderlyShutdown 
      Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to ***** 
      at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741) 
      at hudson.remoting.Request.call(Request.java:202) 
      at hudson.remoting.Channel.call(Channel.java:954) 
      at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller.install(JnlpSlaveRestarterInstaller.java:61) 
      at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller.access$000(JnlpSlaveRestarterInstaller.java:35) 
      at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$Install.call(JnlpSlaveRestarterInstaller.java:49) 
      at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$Install.call(JnlpSlaveRestarterInstaller.java:40) 
      at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) 
      at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71) 
      at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
      at java.lang.Thread.run(Thread.java:748) 
      Caused: hudson.remoting.RequestAbortedException 
      at hudson.remoting.Request.abort(Request.java:340) 
      at hudson.remoting.Channel.terminate(Channel.java:1038) 
      at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1271) 
      at hudson.remoting.Channel$1.handle(Channel.java:565) 
      at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:87)

       

      Around the same time as the error appearing above there is an error on detected by the master (also occurs for free temp space, free disk space, etc.), and then the node is made offline:

       

      Failed to monitor ***** for Response Time 
      java.util.concurrent.TimeoutException 
      at hudson.remoting.Request$1.get(Request.java:316) 
      at hudson.remoting.Request$1.get(Request.java:240) 
      at hudson.remoting.FutureAdapter.get(FutureAdapter.java:59) 
      at hudson.node_monitors.AbstractAsyncNodeMonitorDescriptor.monitor(AbstractAsyncNodeMonitorDescriptor.java:96) 
      at hudson.node_monitors.ResponseTimeMonitor$1.monitor(ResponseTimeMonitor.java:59) 
      at hudson.node_monitors.AbstractNodeMonitorDescriptor$Record.run(AbstractNodeMonitorDescriptor.java:305)

       

      It looks like the closest issue to this is https://issues.jenkins-ci.org/browse/JENKINS-34044

            francisu Francis Upton
            dfrank David Frank
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: