Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-55033

Jenkins attempts to connect to Windows slave before GCP VM is provisioned

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • google-compute-engine-plugin
    • None
    • Jenkins: 2.154
      Google Compute Engine Plugin: 1.0.6

      I've created slave VM image with Packer and configured GCP cloud in Jenkins settings according to this guide.

      When I start job requiring "windows" agent Jenkins starts to provision agents in a loop about every minute until I notice 10+ new agents and cancel the build.

      I've attached one "iteration" of "provision loop", which seems to indicate that Jenkins doesn't honor 300 second launch timeout and prematurely gives up on agent and starts to provision next one.

        1. jenkins.log
          5 kB

          [JENKINS-55033] Jenkins attempts to connect to Windows slave before GCP VM is provisioned

          Rachel Yen added a comment -

          Going through the logs, this is what stands out to me:

          WARNING: Error:
java.lang.NullPointerException
	at com.cloudbees.jenkins.plugins.sshcredentials.SSHAuthenticator.newInstance(SSHAuthenticator.java:191)
	at com.google.jenkins.plugins.computeengine.ComputeEngineWindowsLauncher.authenticateSSH(ComputeEngineWindowsLauncher.java:152)
	at com.google.jenkins.plugins.computeengine.ComputeEngineWindowsLauncher.bootstrap(ComputeEngineWindowsLauncher.java:181)
	at com.google.jenkins.plugins.computeengine.ComputeEngineWindowsLauncher.launch(ComputeEngineWindowsLauncher.java:98)
	at com.google.jenkins.plugins.computeengine.ComputeEngineComputerLauncher.launch(ComputeEngineComputerLauncher.java:127)
	at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:294)
	at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
	at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

          It seems we are having issues with your private key credentials. The only time I've had a similar issue to yours was during integration testing when the credentials created weren't properly registered in the system. Can you verify that you have the ssh credentials[ plugin |https://wiki.jenkins.io/display/JENKINS/SSH+Credentials+Plugin] installed and that you created credentials using this plugin? Otherwise, did you create a username and password credential? One of these must be selected under Configure. 

          Rachel Yen added a comment - Going through the logs, this is what stands out to me: WARNING: Error: java.lang.NullPointerException at com.cloudbees.jenkins.plugins.sshcredentials.SSHAuthenticator.newInstance(SSHAuthenticator.java:191) at com.google.jenkins.plugins.computeengine.ComputeEngineWindowsLauncher.authenticateSSH(ComputeEngineWindowsLauncher.java:152) at com.google.jenkins.plugins.computeengine.ComputeEngineWindowsLauncher.bootstrap(ComputeEngineWindowsLauncher.java:181) at com.google.jenkins.plugins.computeengine.ComputeEngineWindowsLauncher.launch(ComputeEngineWindowsLauncher.java:98) at com.google.jenkins.plugins.computeengine.ComputeEngineComputerLauncher.launch(ComputeEngineComputerLauncher.java:127) at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:294) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang. Thread .run( Thread .java:748) It seems we are having issues with your private key credentials. The only time I've had a similar issue to yours was during integration testing when the credentials created weren't properly registered in the system. Can you verify that you have the ssh credentials[ plugin | https://wiki.jenkins.io/display/JENKINS/SSH+Credentials+Plugin ] installed and that you created credentials using this plugin? Otherwise, did you create a username and password credential? One of these must be selected under Configure. 

          Vadim Gusev added a comment -

          zombiemoose I'm using login-pass pair. I can manually create VM instance and SSH into it with those credentials. I've re-checked that it's correct by re-creating it in Jenkins credential store and re-enabling it in plugin config.

          I've re-created Packer image with actual SSH key and it actually works. So it's password auth that's broken.

          Also aforementioned guide lacks Git resulting in `java.io.IOException: Cannot run program "git" (in directory "C:\JenkinsSlave\workspace\appdater_preloader_develop"): CreateProcess error=2, The system cannot find the file specified`.

          Adding `choco install -y git` to provision script did the trick.

          Vadim Gusev added a comment - zombiemoose I'm using login-pass pair. I can manually create VM instance and SSH into it with those credentials. I've re-checked that it's correct by re-creating it in Jenkins credential store and re-enabling it in plugin config. I've re-created Packer image with actual SSH key and it actually works. So it's password auth that's broken. Also aforementioned guide lacks Git resulting in `java.io.IOException: Cannot run program "git" (in directory "C:\JenkinsSlave\workspace\appdater_preloader_develop"): CreateProcess error=2, The system cannot find the file specified`. Adding `choco install -y git` to provision script did the trick.

          Rachel Yen added a comment -

          Ah, ok, I will need to look into username/password authentication. Thanks for pointing this out.

          That's very interesting, are you using the windows 2016 image? I've never needed to install git separately. That's a very interesting issue I will have to consider to better help our users.

          Rachel Yen added a comment - Ah, ok, I will need to look into username/password authentication. Thanks for pointing this out. That's very interesting, are you using the windows 2016 image? I've never needed to install git separately. That's a very interesting issue I will have to consider to better help our users.

          Vadim Gusev added a comment -

          Yep, I've used exactly the same provision script as in guide. Exact version from Packer logs: googlecompute: Using image: windows-server-2016-dc-v20181113.

          Vadim Gusev added a comment - Yep, I've used exactly the same provision script as in guide. Exact version from Packer logs: googlecompute: Using image: windows-server-2016-dc-v20181113.

          Rachel Yen added a comment -

          I have a commit out that will fix the password issue. Regarding the startup script to install git, I will need to look into that. Will close this when we release my fix.

          Rachel Yen added a comment - I have a commit out that will fix the password issue. Regarding the startup script to install git, I will need to look into that. Will close this when we release my fix.

          Rachel Yen added a comment -

          Just released the fix. Let us know if you have further issues, but I will resolve this one.

          Rachel Yen added a comment - Just released the fix. Let us know if you have further issues, but I will resolve this one.

            zombiemoose Rachel Yen
            zarbis Vadim Gusev
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: