Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-42856

Unable to launch Windows slaves using Microsoft OpenSSH: Unexpected termination of the channel

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Minor Minor
    • ssh-slaves-plugin
    • None
    • Windows 2012 R2 master and slaves, Jenkins 2.32.3 LTS, Win32-OpenSSH 0.0.10.0, SSH Slaves plugin 1.14

      As a first step to being able to use the VMware plugin to spawn slaves automatically, I'm trying to get ssh slave connectivity to work. Normally I use Java Web Start for permanent slaves, which works fine, but I have no way to use that with a dynamic slave (without turning off the connect security and forgoing having a secret in the JNLP command).

      I've installed the Microsoft SSH port (Win32-OpenSSH) for sshd, and Cygwin to get the "true" utility that Jenkins expects to call, but my attempts to launch the agent fail almost immediately (after copying the slave.jar file), with the following log:

      [03/16/17 14:35:26] [SSH] Opening SSH connection to myserver:22.
[03/16/17 14:35:31] [SSH] Authentication successful.
[03/16/17 14:35:34] [SSH] The remote users environment is:
[03/16/17 14:35:36] [SSH] Starting sftp client.
[03/16/17 14:35:39] [SSH] Copying latest slave.jar...
[03/16/17 14:36:00] [SSH] Copied 717,563 bytes.
Expanded the channel window size to 4MB
[03/16/17 14:36:00] [SSH] Starting slave process: cd "C:\Jenkins" && "C:\Program Files\Java\jre1.8.0_121\bin\java"  -jar slave.jar
<===[JENKINS REMOTING CAPACITY]===><===[JENKINS REMOTING CAPACITY]===>ERROR: Unexpected error in launching a slave. This is probably a bug in Jenkins.
hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
	at hudson.remoting.Request.abort(Request.java:307)
	at hudson.remoting.Channel.terminate(Channel.java:888)
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:92)
	at ......remote call to myserver(Native Method)
	at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1537)
	at hudson.remoting.Request.call(Request.java:172)
	at hudson.remoting.Channel.call(Channel.java:821)
	at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:516)
	at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:389)
	at hudson.plugins.sshslaves.SSHLauncher.startSlave(SSHLauncher.java:989)
	at hudson.plugins.sshslaves.SSHLauncher.access$400(SSHLauncher.java:139)
	at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:738)
	at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:719)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)
Caused by: java.io.IOException: Unexpected termination of the channel
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:73)
Caused by: java.io.EOFException
	at java.io.ObjectInputStream$PeekInputStream.readFully(Unknown Source)
	at java.io.ObjectInputStream$BlockDataInputStream.readShort(Unknown Source)
	at java.io.ObjectInputStream.readStreamHeader(Unknown Source)
	at java.io.ObjectInputStream.<init>(Unknown Source)
	at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:48)
	at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:59)
[03/16/17 14:36:09] Launch failed - cleaning up connection
ERROR: Connection terminated
java.io.IOException: Unexpected termination of the channel
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:73)
Caused by: java.io.EOFException
	at java.io.ObjectInputStream$PeekInputStream.readFully(Unknown Source)
	at java.io.ObjectInputStream$BlockDataInputStream.readShort(Unknown Source)
	at java.io.ObjectInputStream.readStreamHeader(Unknown Source)
	at java.io.ObjectInputStream.<init>(Unknown Source)
	at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:48)
	at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:59)
[03/16/17 14:36:09] [SSH] Connection closed.

      I've searched and found a number of cases of the "This is probably a bug in Jenkins." error in launching a slave, but seemingly not this exact one. It seems to be that the connection or process is being terminated on the slave side but I can't figure out by what.

      How can this be fixed or worked around?

      EDIT (March 17, 2017): Per stephenconnolly and my own testing, this appears to be n incompatibility between the Jenkins SSH Slaves plugin and a bug in the Microsoft's Win32-OpenSSH port specifically. When I switched the slave node to run Cygwin's OpenSSH, it worked perfectly.

        1. sshlogMaster.log
          5 kB
        2. sshlogMaster.log
          5 kB
        3. Screen Shot 2017-03-17 at 20.59.21.png
          Screen Shot 2017-03-17 at 20.59.21.png
          44 kB
        4. image-2017-03-16-22-26-53-920.png
          image-2017-03-16-22-26-53-920.png
          71 kB

          [JENKINS-42856] Unable to launch Windows slaves using Microsoft OpenSSH: Unexpected termination of the channel

          Nick Jones created issue -
          Nick Jones made changes -
          Issue Type Original: Improvement [ 4 ] New: Bug [ 1 ]
          Nick Jones made changes -
          Priority Original: Major [ 3 ] New: Blocker [ 1 ]
          Nick Jones made changes -
          Environment Original: Windows 2012 R2 master and slaves, Jenkins 2.32.3 LTS, Win32-OpenSSH 0.0.10.0, Cygwin 2.7.0 New: Windows 2012 R2 master and slaves, Jenkins 2.32.3 LTS, Win32-OpenSSH 0.0.10.0, Cygwin 2.7.0, SSH Slaves plugin 1.14

          Nick Jones added a comment -

          I just updated to the latest version of the SSH Slaves plugin, 1.14, and tried again. The only difference is that now it correctly outputs the remote user's environment. It still fails immediately after slave.jar copying with:

          ERROR: Unexpected error in launching a slave. This is probably a bug in Jenkins.
hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
	at hudson.remoting.Request.abort(Request.java:307)
	at hudson.remoting.Channel.terminate(Channel.java:888)

          etc.

          Nick Jones added a comment - I just updated to the latest version of the SSH Slaves plugin, 1.14, and tried again. The only difference is that now it correctly outputs the remote user's environment. It still fails immediately after slave.jar copying with: ERROR: Unexpected error in launching a slave. This is probably a bug in Jenkins. hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Request.abort(Request.java:307) at hudson.remoting.Channel.terminate(Channel.java:888) etc.
          Nick Jones made changes -
          Description Original: As a first step to being able to use the VMware plugin to spawn slaves automatically, I'm trying to get ssh slave connectivity to work. Normally I use Java Web Start for permanent slaves, which works fine, but I have no way to use that with a dynamic slave (without turning off the connect security and forgoing having a secret in the JNLP command).

          I've installed the Microsoft SSH port (Win32-OpenSSH) for sshd, and Cygwin to get the "true" utility that Jenkins expects to call, but my attempts to launch the agent fail almost immediately (after copying the slave.jar file), with the following log:
          {noformat}
          [03/16/17 14:35:26] [SSH] Opening SSH connection to myserver:22.
          [03/16/17 14:35:31] [SSH] Authentication successful.
          [03/16/17 14:35:34] [SSH] The remote users environment is:
          [03/16/17 14:35:36] [SSH] Starting sftp client.
          [03/16/17 14:35:39] [SSH] Copying latest slave.jar...
          [03/16/17 14:36:00] [SSH] Copied 717,563 bytes.
          Expanded the channel window size to 4MB
          [03/16/17 14:36:00] [SSH] Starting slave process: cd "C:\Jenkins" && "C:\Program Files\Java\jre1.8.0_121\bin\java" -jar slave.jar
          <===[JENKINS REMOTING CAPACITY]===><===[JENKINS REMOTING CAPACITY]===>ERROR: Unexpected error in launching a slave. This is probably a bug in Jenkins.
          hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
          at hudson.remoting.Request.abort(Request.java:307)
          at hudson.remoting.Channel.terminate(Channel.java:888)
          at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:92)
          at ......remote call to BuildBase(Native Method)
          at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1537)
          at hudson.remoting.Request.call(Request.java:172)
          at hudson.remoting.Channel.call(Channel.java:821)
          at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:516)
          at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:389)
          at hudson.plugins.sshslaves.SSHLauncher.startSlave(SSHLauncher.java:989)
          at hudson.plugins.sshslaves.SSHLauncher.access$400(SSHLauncher.java:139)
          at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:738)
          at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:719)
          at java.util.concurrent.FutureTask.run(Unknown Source)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
          at java.lang.Thread.run(Unknown Source)
          Caused by: java.io.IOException: Unexpected termination of the channel
          at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:73)
          Caused by: java.io.EOFException
          at java.io.ObjectInputStream$PeekInputStream.readFully(Unknown Source)
          at java.io.ObjectInputStream$BlockDataInputStream.readShort(Unknown Source)
          at java.io.ObjectInputStream.readStreamHeader(Unknown Source)
          at java.io.ObjectInputStream.<init>(Unknown Source)
          at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:48)
          at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
          at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:59)
          [03/16/17 14:36:09] Launch failed - cleaning up connection
          ERROR: Connection terminated
          java.io.IOException: Unexpected termination of the channel
          at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:73)
          Caused by: java.io.EOFException
          at java.io.ObjectInputStream$PeekInputStream.readFully(Unknown Source)
          at java.io.ObjectInputStream$BlockDataInputStream.readShort(Unknown Source)
          at java.io.ObjectInputStream.readStreamHeader(Unknown Source)
          at java.io.ObjectInputStream.<init>(Unknown Source)
          at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:48)
          at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
          at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:59)
          [03/16/17 14:36:09] [SSH] Connection closed.{noformat}
          I've searched and found a number of cases of the "This is probably a bug in Jenkins." error in launching a slave, but seemingly not this exact one. It seems to be that the connection or process is being terminated on the slave side but I can't figure out by what.

          How can this be fixed or worked around?

                      		New: As a first step to being able to use the VMware plugin to spawn slaves automatically, I'm trying to get ssh slave connectivity to work. Normally I use Java Web Start for permanent slaves, which works fine, but I have no way to use that with a dynamic slave (without turning off the connect security and forgoing having a secret in the JNLP command).

          I've installed the Microsoft SSH port (Win32-OpenSSH) for sshd, and Cygwin to get the "true" utility that Jenkins expects to call, but my attempts to launch the agent fail almost immediately (after copying the slave.jar file), with the following log:
          {noformat}
          [03/16/17 14:35:26] [SSH] Opening SSH connection to myserver:22.
          [03/16/17 14:35:31] [SSH] Authentication successful.
          [03/16/17 14:35:34] [SSH] The remote users environment is:
          [03/16/17 14:35:36] [SSH] Starting sftp client.
          [03/16/17 14:35:39] [SSH] Copying latest slave.jar...
          [03/16/17 14:36:00] [SSH] Copied 717,563 bytes.
          Expanded the channel window size to 4MB
          [03/16/17 14:36:00] [SSH] Starting slave process: cd "C:\Jenkins" && "C:\Program Files\Java\jre1.8.0_121\bin\java" -jar slave.jar
          <===[JENKINS REMOTING CAPACITY]===><===[JENKINS REMOTING CAPACITY]===>ERROR: Unexpected error in launching a slave. This is probably a bug in Jenkins.
          hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
          at hudson.remoting.Request.abort(Request.java:307)
          at hudson.remoting.Channel.terminate(Channel.java:888)
          at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:92)
          at ......remote call to myserver(Native Method)
          at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1537)
          at hudson.remoting.Request.call(Request.java:172)
          at hudson.remoting.Channel.call(Channel.java:821)
          at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:516)
          at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:389)
          at hudson.plugins.sshslaves.SSHLauncher.startSlave(SSHLauncher.java:989)
          at hudson.plugins.sshslaves.SSHLauncher.access$400(SSHLauncher.java:139)
          at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:738)
          at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:719)
          at java.util.concurrent.FutureTask.run(Unknown Source)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
          at java.lang.Thread.run(Unknown Source)
          Caused by: java.io.IOException: Unexpected termination of the channel
          at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:73)
          Caused by: java.io.EOFException
          at java.io.ObjectInputStream$PeekInputStream.readFully(Unknown Source)
          at java.io.ObjectInputStream$BlockDataInputStream.readShort(Unknown Source)
          at java.io.ObjectInputStream.readStreamHeader(Unknown Source)
          at java.io.ObjectInputStream.<init>(Unknown Source)
          at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:48)
          at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
          at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:59)
          [03/16/17 14:36:09] Launch failed - cleaning up connection
          ERROR: Connection terminated
          java.io.IOException: Unexpected termination of the channel
          at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:73)
          Caused by: java.io.EOFException
          at java.io.ObjectInputStream$PeekInputStream.readFully(Unknown Source)
          at java.io.ObjectInputStream$BlockDataInputStream.readShort(Unknown Source)
          at java.io.ObjectInputStream.readStreamHeader(Unknown Source)
          at java.io.ObjectInputStream.<init>(Unknown Source)
          at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:48)
          at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
          at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:59)
          [03/16/17 14:36:09] [SSH] Connection closed.{noformat}
          I've searched and found a number of cases of the "This is probably a bug in Jenkins." error in launching a slave, but seemingly not this exact one. It seems to be that the connection or process is being terminated on the slave side but I can't figure out by what.

          How can this be fixed or worked around?

           

          Oleg Nenashev added a comment -

          CC stephenconnolly since he likely has some context

          Oleg Nenashev added a comment - CC stephenconnolly since he likely has some context
          Stephen Connolly made changes -
          Attachment New: image-2017-03-16-22-26-53-920.png [ 36566 ]

          Stephen Connolly added a comment -

          The Microsoft OpenSSH port has some issue that I - after 2 days fighting - was unable to get around.

          There is currently no way to make this work with Jenkins.

          It does not work even using the `use a command on the command line` option to fire a native ssh client.

          I am not sure if it is CHCP code page related mangling of the streams (though ssh -T should avoid that, and that didn't work)

          I wish I had even a reproducible test case that I could provide to Microsoft to raise the issue.

          The state of affairs results in me having to post this here: 

          Stephen Connolly added a comment - The Microsoft OpenSSH port has some issue that I - after 2 days fighting - was unable to get around. There is currently no way to make this work with Jenkins. It does not work even using the `use a command on the command line` option to fire a native ssh client. I am not sure if it is CHCP code page related mangling of the streams (though ssh -T should avoid that, and that didn't work) I wish I had even a reproducible test case that I could provide to Microsoft to raise the issue. The state of affairs results in me having to post this here: 

          Nick Jones added a comment - - edited

          Thanks stephenc. Taking a step back, is there any sshd that has been found to work with Jenkins on Windows, if not the Microsoft OpenSSH one?

          Nick Jones added a comment - - edited Thanks stephenc . Taking a step back, is there any sshd that has been found to work with Jenkins on Windows, if not the Microsoft OpenSSH one?

            ifernandezcalvo Ivan Fernandez Calvo
            medianick Nick Jones
            Votes:
            5 Vote for this issue
            Watchers:
            14 Start watching this issue

              Created:
              Updated:
              Resolved: