Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-19758

Jenkins: [reproducible] ssh slave log not available, spinning wheel after upgrading to SSH Credentials 1.4, SSH Slaves 1.2 and disconnecting/reconnecting

      With a brand new Jenkins 1.532 install, no plugins, I am able to setup a dumb ssh slave fine, and can see the slave logfile when I disconnect/reconnect and click Log. This is using Credentials Plugin 1.4, SSH Credentials Plugin 0.3, SSH Slaves Plugin 0.25

      When I update to use Credentials Plugin 1.8.2 and restart, this process continues working fine. So far so good.

      But when I update to use SSH Credentials Plugin 1.4, SSH Slaves Plugin 1.2 I see a regression – when I disconnect/reconnect, then click on Log to view the slave logfile, I just see a spinning wheel in the browser. I can tell the slave did successfully connect (since I see 1 executor available), but I cannot see any slave log output.

      I can consistently reproduce this issue.

      This is a pretty major issue when used in a virtualized slaves environment where slaves are disconnected, reset, and reconnected, since this makes it very difficult to understand the status of the slaves and to troubleshoot slave connection issues.

          [JENKINS-19758] Jenkins: [reproducible] ssh slave log not available, spinning wheel after upgrading to SSH Credentials 1.4, SSH Slaves 1.2 and disconnecting/reconnecting

          Steve Roth created issue -

          Steve Roth added a comment - - edited

          Forgot to mention – this is a (dumb) ssh slave. The SSH credentials are 'username with password' global credentials.

          Steve Roth added a comment - - edited Forgot to mention – this is a (dumb) ssh slave. The SSH credentials are 'username with password' global credentials.

          Steve Roth added a comment -

          The slave logfile on the master, at logs/slaves/<someslave> is also not written when the reconnect occurs. The original logfile is moved to slave.log.1, but there is no slave.log file.

          The master logfile does not report any error:
          Sep 25, 2013 10:08:39 AM hudson.slaves.SlaveComputer tryReconnect
          INFO: Attempting to reconnect someslavebox

          Steve Roth added a comment - The slave logfile on the master, at logs/slaves/<someslave> is also not written when the reconnect occurs. The original logfile is moved to slave.log.1, but there is no slave.log file. The master logfile does not report any error: Sep 25, 2013 10:08:39 AM hudson.slaves.SlaveComputer tryReconnect INFO: Attempting to reconnect someslavebox

          Steve Roth added a comment -

          It's interesting that (after updating these plugin versions), and doing an initial Jenkins restart, I am able to see the Slave logfile by clicking on Log. But if I then disconnect and reconnect the slave, I now see the spinning wheel and no logfile.

          So this implies to me that the slave connection process is slightly different between the initial startup and the manual disconnect/reconnect.

          Steve Roth added a comment - It's interesting that (after updating these plugin versions), and doing an initial Jenkins restart, I am able to see the Slave logfile by clicking on Log. But if I then disconnect and reconnect the slave, I now see the spinning wheel and no logfile. So this implies to me that the slave connection process is slightly different between the initial startup and the manual disconnect/reconnect.
          Steve Roth made changes -
          Description Original: With a brand new Jenkins 1.532 install, no plugins, I am able to setup a slave fine, and see the slave log as I disconnect/reconnect. This is using Credentials Plugin 1.4, SSH Credentials Plugin 0.3, SSH Slaves Plugin 0.25

          When I update to use Credentials Plugin 1.8.2 and restart, everything continues working fine.

          But when I update to use SSH Credentials Plugin 1.4, SSH Slaves Plugin 1.2 I see a regression -- when I disconnect/reconnect, then click on Log to view the slave logfile, I just see a spinning wheel in the browser. I can tell the slave did successfully connect (since I see 1 executor available), but I cannot see any slave output.

          This is a pretty major issue when used in a virtualized slaves environment, since this makes it very difficult to understand the status of the slaves and to troubleshoot slave connection issues.
          New: With a brand new Jenkins 1.532 install, no plugins, I am able to setup a slave fine, and can see the slave logfile when I disconnect/reconnect. This is using Credentials Plugin 1.4, SSH Credentials Plugin 0.3, SSH Slaves Plugin 0.25

          When I update to use Credentials Plugin 1.8.2 and restart, this process continues working fine.

          But when I update to use SSH Credentials Plugin 1.4, SSH Slaves Plugin 1.2 I see a regression -- when I disconnect/reconnect, then click on Log to view the slave logfile, I just see a spinning wheel in the browser. I can tell the slave did successfully connect (since I see 1 executor available), but I cannot see any slave log output.

          I can consistently reproduce this issue.

          This is a pretty major issue when used in a virtualized slaves environment, since this makes it very difficult to understand the status of the slaves and to troubleshoot slave connection issues.
          Summary Original: Jenkins: ssh slave log not available, spinning wheel after upgrading to SSH Credentials 1.4, SSH Slaves 1.2 New: Jenkins: [reproducible] ssh slave log not available, spinning wheel after upgrading to SSH Credentials 1.4, SSH Slaves 1.2 and disconnecting/reconnecting
          Steve Roth made changes -
          Description Original: With a brand new Jenkins 1.532 install, no plugins, I am able to setup a slave fine, and can see the slave logfile when I disconnect/reconnect. This is using Credentials Plugin 1.4, SSH Credentials Plugin 0.3, SSH Slaves Plugin 0.25

          When I update to use Credentials Plugin 1.8.2 and restart, this process continues working fine.

          But when I update to use SSH Credentials Plugin 1.4, SSH Slaves Plugin 1.2 I see a regression -- when I disconnect/reconnect, then click on Log to view the slave logfile, I just see a spinning wheel in the browser. I can tell the slave did successfully connect (since I see 1 executor available), but I cannot see any slave log output.

          I can consistently reproduce this issue.

          This is a pretty major issue when used in a virtualized slaves environment, since this makes it very difficult to understand the status of the slaves and to troubleshoot slave connection issues.
          New: With a brand new Jenkins 1.532 install, no plugins, I am able to setup a dumb ssh slave fine, and can see the slave logfile when I disconnect/reconnect and click Log. This is using Credentials Plugin 1.4, SSH Credentials Plugin 0.3, SSH Slaves Plugin 0.25

          When I update to use Credentials Plugin 1.8.2 and restart, this process continues working fine. So far so good.

          But when I update to use SSH Credentials Plugin 1.4, SSH Slaves Plugin 1.2 I see a regression -- when I disconnect/reconnect, then click on Log to view the slave logfile, I just see a spinning wheel in the browser. I can tell the slave did successfully connect (since I see 1 executor available), but I cannot see any slave log output.

          I can consistently reproduce this issue.

          This is a pretty major issue when used in a virtualized slaves environment, since this makes it very difficult to understand the status of the slaves and to troubleshoot slave connection issues.
          Steve Roth made changes -
          Description Original: With a brand new Jenkins 1.532 install, no plugins, I am able to setup a dumb ssh slave fine, and can see the slave logfile when I disconnect/reconnect and click Log. This is using Credentials Plugin 1.4, SSH Credentials Plugin 0.3, SSH Slaves Plugin 0.25

          When I update to use Credentials Plugin 1.8.2 and restart, this process continues working fine. So far so good.

          But when I update to use SSH Credentials Plugin 1.4, SSH Slaves Plugin 1.2 I see a regression -- when I disconnect/reconnect, then click on Log to view the slave logfile, I just see a spinning wheel in the browser. I can tell the slave did successfully connect (since I see 1 executor available), but I cannot see any slave log output.

          I can consistently reproduce this issue.

          This is a pretty major issue when used in a virtualized slaves environment, since this makes it very difficult to understand the status of the slaves and to troubleshoot slave connection issues.
          New: With a brand new Jenkins 1.532 install, no plugins, I am able to setup a dumb ssh slave fine, and can see the slave logfile when I disconnect/reconnect and click Log. This is using Credentials Plugin 1.4, SSH Credentials Plugin 0.3, SSH Slaves Plugin 0.25

          When I update to use Credentials Plugin 1.8.2 and restart, this process continues working fine. So far so good.

          But when I update to use SSH Credentials Plugin 1.4, SSH Slaves Plugin 1.2 I see a regression -- when I disconnect/reconnect, then click on Log to view the slave logfile, I just see a spinning wheel in the browser. I can tell the slave did successfully connect (since I see 1 executor available), but I cannot see any slave log output.

          I can consistently reproduce this issue.

          This is a pretty major issue when used in a virtualized slaves environment where slaves are disconnected, reset, and reconnected, since this makes it very difficult to understand the status of the slaves and to troubleshoot slave connection issues.

          Steve Roth added a comment - - edited

          Possibly related – though I can repro this issue in the latest and greatest Jenkins (1.532), I am also seeing it if we update our 1.518 instance (with Credentials 1.8.2, SSH credentials 1.4, SSH slaves 1.2) to 1.519 or later.

          Using the latest Credentials, SSH Credentials, and SSH Slaves versions, I have tried many different versions of Jenkins, starting with 1.519, and all of them have the same issue. But if I switch back to 1.518, the issue goes away.

          So to me that implies that 1.518 works with the newer versions of the SSH Credentials and SSH slaves plugins, but 1.519 (and later) do not. Something must have changed in the 1.519 codebase.

          So we are currently stuck on 1.518 till this issue is resolved. This is the only issue preventing us from moving to more recent Jenkins versions.

          Steve Roth added a comment - - edited Possibly related – though I can repro this issue in the latest and greatest Jenkins (1.532), I am also seeing it if we update our 1.518 instance (with Credentials 1.8.2, SSH credentials 1.4, SSH slaves 1.2) to 1.519 or later. Using the latest Credentials, SSH Credentials, and SSH Slaves versions, I have tried many different versions of Jenkins, starting with 1.519, and all of them have the same issue. But if I switch back to 1.518, the issue goes away. So to me that implies that 1.518 works with the newer versions of the SSH Credentials and SSH slaves plugins, but 1.519 (and later) do not. Something must have changed in the 1.519 codebase. So we are currently stuck on 1.518 till this issue is resolved. This is the only issue preventing us from moving to more recent Jenkins versions.

          1.533 does not show slave log in web page.
          On jenkins/logs/slaves/<SLAVE>/ there is no slave.log only slave.log.NNN where NNN is some number like .10 or .2 or .5 it seems to be random.

          Corneil du Plessis added a comment - 1.533 does not show slave log in web page. On jenkins/logs/slaves/<SLAVE>/ there is no slave.log only slave.log.NNN where NNN is some number like .10 or .2 or .5 it seems to be random.

          Stephen Connolly added a comment - - edited

          My working theory is that https://github.com/jenkinsci/jenkins/commit/d10fbc5eace959d3a5928d40b1c1f8975b3be1d0 is to blame

          Update: this theory proved incorrect when bug not reproduced in Jenkins 1.519

          Stephen Connolly added a comment - - edited My working theory is that https://github.com/jenkinsci/jenkins/commit/d10fbc5eace959d3a5928d40b1c1f8975b3be1d0 is to blame Update : this theory proved incorrect when bug not reproduced in Jenkins 1.519

            stephenconnolly Stephen Connolly
            sroth Steve Roth
            Votes:
            4 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved: