Loading...

Type: Bug
Resolution: Cannot Reproduce
Priority: Minor
Component/s: ssh-slaves-plugin
Labels:
None
Environment:

Hide
jenkins 2.32.2 on ubuntu 14.04
slave is an ARM based embedded system with OpenSSH, running
java version "1.7.0_75"
Java(TM) SE Embedded Runtime Environment (build 1.7.0_75-b13, headless)
Java HotSpot(TM) Embedded Client VM (build 24.75-b04, mixed mode)

Slave's /etc/ssh/sshd_config contains
ClientAliveInterval 15
ClientAliveCountMax 4

Show
jenkins 2.32.2 on ubuntu 14.04 slave is an ARM based embedded system with OpenSSH, running java version "1.7.0_75" Java(TM) SE Embedded Runtime Environment (build 1.7.0_75-b13, headless) Java HotSpot(TM) Embedded Client VM (build 24.75-b04, mixed mode) Slave's /etc/ssh/sshd_config contains ClientAliveInterval 15 ClientAliveCountMax 4

Similar Issues:

Show

Slave connection fails shortly after etablishing. After some time period it retries and connects again, then fails again. Repeat ad-infinitum. Connect doe not fail during a build if the build starts while the connection is up- but the connection then fails after the build finishes.

Slave log

03/03/17 09:37:25] [SSH] Opening SSH connection to 10.42.0.92:22.

[03/03/17 09:37:25] [SSH] Authentication successful.
[03/03/17 09:37:25] [SSH] The remote users environment is:
BASH=/bin/sh
BASHOPTS=cmdhist:complete_fullquote:extquote:force_fignore:hostcomplete:interactive_comments:progcomp:promptvars:sourcepath
BASH_ALIASES=()
BASH_ARGC=()
BASH_ARGV=()
BASH_CMDS=()
BASH_EXECUTION_STRING=set
BASH_LINENO=()
BASH_SOURCE=()
BASH_VERSINFO=([0]="4" [1]="3" [2]="39" [3]="1" [4]="release" [5]="arm-dey-linux-gnueabi")
BASH_VERSION='4.3.39(1)-release'
DIRSTACK=()
EUID=0
GROUPS=()
HOME=/home/root
HOSTNAME=marianos_kite
HOSTTYPE=arm
IFS=' 	
'
LOGNAME=root
MACHTYPE=arm-dey-linux-gnueabi
MAIL=/var/mail/root
OPTERR=1
OPTIND=1
OSTYPE=linux-gnueabi
PATH=/usr/bin:/bin:/usr/sbin:/sbin
POSIXLY_CORRECT=y
PPID=13150
PS4='+ '
PWD=/home/root
SHELL=/bin/sh
SHELLOPTS=braceexpand:hashall:interactive-comments:posix
SHLVL=1
SSH_CLIENT='10.42.0.1 40830 22'
SSH_CONNECTION='10.42.0.1 40830 10.42.0.92 22'
TERM=dumb
TZ=UTC
UID=0
USER=root
_=sh
[03/03/17 09:37:25] [SSH] Checking java version of java
[03/03/17 09:37:26] [SSH] java -version returned 1.7.0_75.
[03/03/17 09:37:26] [SSH] Starting sftp client.
ERROR: [03/03/17 09:37:26] [SSH] SFTP failed. Copying via SCP.
java.io.IOException: Unexpected end of sftp stream.
	at com.trilead.ssh2.SFTPv3Client.readBytes(SFTPv3Client.java:217)
	at com.trilead.ssh2.SFTPv3Client.receiveMessage(SFTPv3Client.java:240)
	at com.trilead.ssh2.SFTPv3Client.init(SFTPv3Client.java:864)
	at com.trilead.ssh2.SFTPv3Client.<init>(SFTPv3Client.java:108)
	at com.trilead.ssh2.SFTPv3Client.<init>(SFTPv3Client.java:119)
	at hudson.plugins.sshslaves.SFTPClient.<init>(SFTPClient.java:45)
	at hudson.plugins.sshslaves.SSHLauncher.copySlaveJar(SSHLauncher.java:1028)
	at hudson.plugins.sshslaves.SSHLauncher.access$300(SSHLauncher.java:139)
	at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:736)
	at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:719)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
[03/03/17 09:37:26] [SSH] Copying latest slave.jar...
Expanded the channel window size to 4MB
[03/03/17 09:37:26] [SSH] Starting slave process: cd "/home/root/jenkins" && java  -jar slave.jar -slaveLog /var/log/jenkinsslave

<===[JENKINS REMOTING CAPACITY]===>channel started

Slave.jar version: 3.4.1
This is a Unix agent

Evacuated stdout
Agent successfully connected and online

ERROR: Connection terminated
java.io.IOException: Unexpected termination of the channel
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:73)
Caused by: java.io.EOFException
	at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2620)
	at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3095)
	at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:851)
	at java.io.ObjectInputStream.<init>(ObjectInputStream.java:348)
	at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:48)
	at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:59)
ERROR: Socket connection to SSH server was lost
java.io.IOException: Unexpected SSH_MSG_CHANNEL_REQUEST message for non-existent channel 103
	at com.trilead.ssh2.channel.ChannelManager.msgChannelRequest(ChannelManager.java:1144)
	at com.trilead.ssh2.channel.ChannelManager.handleMessage(ChannelManager.java:1466)
	at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:796)
	at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489)
	at java.lang.Thread.run(Thread.java:745)
Slave JVM has not reported exit code before the socket was lost
[03/03/17 09:37:51] [SSH] Connection closed.

[03/03/17 09:38:20] [SSH] Opening SSH connection to 10.42.0.92:22.
[03/03/17 09:38:21] [SSH] Authentication successful.
[03/03/17 09:38:21] [SSH] The remote users environment is:

at which point it basically repeats.

Workaround:

On the slave, comment out the

ClientAliveInterval 15
ClientAliveCountMax 4

lines in

/etc/ssh/sshd_config

and restart sshd, slave will now connect and stay connected next time it retries.

Sorry but I do not have a more conventional (x86 linux) slave that I can test with in my environment to see if this is a peculiarity of the ARM based slave environment, or general. My hunch is it will be general.

Details

Description

Workaround:

Attachments

Activity

People

Dates