Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-17003

JNLP Slave on Windows Server 2012 will not reconnect to master after network disconnect/reconnect

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Blocker Blocker
    • remoting
    • Master running Jenkins 1.460 on Windows Server 2008 R2, slave running JNLP as Windows service on Windows Server 2012

      I have a JNLP slave running on a remote Windows server. Due to the fact that communication goes over the internet, at times the connectivity between the master and the slave will be broken.

      When connectivity is reestablished though the slave will not try to reconnect with the master. A reset of the Windows service hosting the slave re-established connection and the slave becomes available in the master.

      An error message is recorded here. This error as far as I can see occurs at the time network connectivity is lost, and the "INFO: Terminated" shows that slave indeed have "died".
      NOTE: even though the Java code has terminated, the Windows service is still running.

      feb 17, 2013 3:33:18 AM hudson.remoting.jnlp.Main$CuiListener <init>
      INFO: Hudson agent is running in headless mode.
      feb 17, 2013 3:33:18 AM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Locating server among https://externalyavailabledomain/build/, http://internalmasterserver:8080/build/
      feb 17, 2013 3:33:19 AM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Connecting to externalyavailabledomain:50456
      feb 17, 2013 3:33:19 AM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Handshaking
      feb 17, 2013 3:33:19 AM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Connected
      feb 17, 2013 1:59:20 PM hudson.remoting.SynchronousCommandTransport$ReaderThread run
      SEVERE: I/O error in channel channel
      java.net.SocketException: Connection reset
      at java.net.SocketInputStream.read(Unknown Source)
      at java.net.SocketInputStream.read(Unknown Source)
      at java.io.BufferedInputStream.fill(Unknown Source)
      at java.io.BufferedInputStream.read(Unknown Source)
      at java.io.ObjectInputStream$PeekInputStream.peek(Unknown Source)
      at java.io.ObjectInputStream$BlockDataInputStream.peek(Unknown Source)
      at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)
      at java.io.ObjectInputStream.readObject0(Unknown Source)
      at java.io.ObjectInputStream.readObject(Unknown Source)
      at hudson.remoting.Command.readFrom(Command.java:90)
      at hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59)
      at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

      feb 17, 2013 1:59:20 PM hudson.remoting.jnlp.Main$CuiListener status
      INFO: Terminated

          [JENKINS-17003] JNLP Slave on Windows Server 2012 will not reconnect to master after network disconnect/reconnect

          Note: I have rated this issue as a Blocker since it effectively makes slaves communicating over unreliable networks unusable.

          Peter Lillevold added a comment - Note: I have rated this issue as a Blocker since it effectively makes slaves communicating over unreliable networks unusable.

          Linards L added a comment -

          Having same behaviourt on Wndows 2008 Datacenter R2 SP1.

          Linking the issue(s) to feedback.

          Linards L added a comment - Having same behaviourt on Wndows 2008 Datacenter R2 SP1. Linking the issue(s) to feedback.

          Linards, please add your vote to this issue to raise awareness

          Peter Lillevold added a comment - Linards, please add your vote to this issue to raise awareness

          Linards L added a comment -

          Roger that, Peter!

          Linards L added a comment - Roger that, Peter!

          For those interested, here's a powershell snippet that will start/restart the Jenkins slave service in case of termination: (NOTE: replace the $jenkins variable with the name of your Windows service name)

          $DebugPreference = 2
          $VerbosePreference = 2
          $WarningPreference = 2

          $jenkins = "jenkinsslave-c__Jenkins"
          $now = (Get-Date).ToString()
          $jenkinsService = Get-Service | Where-Object

          {($_.Name -eq $jenkins) -and ($_.Status -eq [System.ServiceProcess.ServiceControllerStatus]::Stopped)}

          if ($jenkinsService.count -eq 1)
          {
          Write-Warning ($now + " Jenkins slave not running. Starting...")
          Start-Service $jenkins
          }
          else
          {

          $errorLog = (Get-Content C:\Jenkins\jenkins-slave.err.log)[-1]
          if ($errorLog.ToLower().CompareTo("info: terminated") -eq 0)

          { Write-Warning ($now + " Jenkins was terminated. Restarting...") Restart-Service $jenkins }

          else

          { Write-Verbose ($now + " Jenkins slave is live and kicking!") }

          }

          Peter Lillevold added a comment - For those interested, here's a powershell snippet that will start/restart the Jenkins slave service in case of termination: (NOTE: replace the $jenkins variable with the name of your Windows service name) $DebugPreference = 2 $VerbosePreference = 2 $WarningPreference = 2 $jenkins = "jenkinsslave-c__Jenkins" $now = (Get-Date).ToString() $jenkinsService = Get-Service | Where-Object {($_.Name -eq $jenkins) -and ($_.Status -eq [System.ServiceProcess.ServiceControllerStatus]::Stopped)} if ($jenkinsService.count -eq 1) { Write-Warning ($now + " Jenkins slave not running. Starting...") Start-Service $jenkins } else { $errorLog = (Get-Content C:\Jenkins\jenkins-slave.err.log) [-1] if ($errorLog.ToLower().CompareTo("info: terminated") -eq 0) { Write-Warning ($now + " Jenkins was terminated. Restarting...") Restart-Service $jenkins } else { Write-Verbose ($now + " Jenkins slave is live and kicking!") } }

          Linards L added a comment -

          Jenkins v1.502 , created slave, using as windows service -> v1.504 , slave working correctly after upgrade.

          Now crossing fingers that upgrade from v1.494 will also result in working slave.

          Linards L added a comment - Jenkins v1.502 , created slave, using as windows service -> v1.504 , slave working correctly after upgrade. Now crossing fingers that upgrade from v1.494 will also result in working slave.

          Linards L added a comment -

          Nop. Still failing.

          Linards L added a comment - Nop. Still failing.

          evernat added a comment -

          Is it reproduced with a recent Jenkins version?

          evernat added a comment - Is it reproduced with a recent Jenkins version?

          Yes, I still need my external monitoring service to keep the slave from going dark, now on Jenkins 1.545 and Slave.jar version 2.33.

          Peter Lillevold added a comment - Yes, I still need my external monitoring service to keep the slave from going dark, now on Jenkins 1.545 and Slave.jar version 2.33.

          Oleg Nenashev added a comment -

          I cannot see how this behavior could happen on the latest release with WinSW 2.0+ and Remoting 3.x. Both components have been improved significantly in terms of the stability. If you still see this issue, please provide logs from the new Jenkins installation and reopen the issue

          Oleg Nenashev added a comment - I cannot see how this behavior could happen on the latest release with WinSW 2.0+ and Remoting 3.x. Both components have been improved significantly in terms of the stability. If you still see this issue, please provide logs from the new Jenkins installation and reopen the issue

            kohsuke Kohsuke Kawaguchi
            plillevold Peter Lillevold
            Votes:
            6 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: