Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-75190

Improve support for connecting to Windows agents using SSH

    • Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Major Major
    • ssh-slaves-plugin
    • None
    • Windows Server 2019 with Open SSH service enabled and running

      Overview

      Please improve the SSH agent integration so that it works with the Open SSH server that ships with Windows 2019 and later. I noticed when trying to get Jenkins to connect via SSH, it assumes that it is connecting to a *nix shell on the other side. I see commands like set, testcmd && cmd expressions, which has varying degrees of support in Windows shells (cmd.exe, powershell.exe, and pwsh.exe). I was able to get an agent working using pwsh.exe as the remote shell, since it supports the cmd && cmd expressions necessary.

      When configuring the Node in the Jenkins server, I should be able to tell Jenkins what shell is being used on the other end, or even better, Jenkins should detect the shell on the other end and issue appropriate commands based on that shell instead of assuming a *nix sh is on the other side.

      And no, I'm not going to install cygwin or other nonsense. This is a Windows machine. Windows now ships with and supports SSH. Jenkins should get updated to reflect that.

      If someone can point me in the direction of source code, I might be able to contribue a PR to fix/improve.

      I've attached the log files I get when connecting to Windows and Open SSH on the Windows server is configured to use cmd.exe, powershell.exe, and pwsh.exe as the remote shell.

      cmd.exe Problems

      The test command fails

      'test' is not recognized as an internal or external command,
      operable program or batch file. 

      Not sure how to test that a file/directory exists in cmd.exe.

      Fails to detect that remote file system already exists and always tries to create it

      [01/24/25 13:53:48] [SSH] Remote file system root D:\J does not exist. Will try to create it...
      A subdirectory or file D:\J already exists.
      Error occurred while processing: D:\J.
      Failed to create D:\J
      

      It looks like the agent is using mkdir -p to create a directory. Windows doesn't have the -p option; it always creates missing parent directories.

      Error starting agent process

      [01/24/25 13:53:49] [SSH] Starting agent process: cd "D:\J" && java  -jar remoting.jar -workDir D:\J -jar-cache D:\J/remoting/jarCache
      Error: Unable to access jarfile remoting.jar
      Agent JVM has terminated. Exit code=1
      

      This is failing because we run our builds on their own D: drive and the cd command in Windows keeps a different working directory per drive. The solution is to pass the /D switch to the cd command: cd /D "D:\J" && java  -jar remoting.jar -workDir D:\J -jar-cache D:\J/remoting/jarCache. It's fine to always pass that switch, even if not switching drives.

      powershell.exe Problems

      The set command fails

      [01/24/25 13:52:22] [SSH] The remote user's environment is:
      Set-Variable : Cannot process command because of one or more missing mandatory parameters: Name.
      At line:1 char:1
      + set
      + ~~~
          + CategoryInfo          : InvalidArgument: (:) [Set-Variable], ParameterBindingException
          + FullyQualifiedErrorId : MissingMandatoryParameter,Microsoft.PowerShell.Commands.SetVariable 
         Command 

      The correct way to display environment variables in PowerShell is to run the Get-ChildItem -Path 'env:' command.1

      The test command fails

      test : The term 'test' is not recognized as the name of a cmdlet, function, script file, or 
      operable program. Check the spelling of the name, or if a path was included, verify that the path 
      is correct and try again.
      At line:1 char:1
      + test -d D:\J
      + ~~~~
          + CategoryInfo          : ObjectNotFound: (test:String) [], CommandNotFoundException
          + FullyQualifiedErrorId : CommandNotFoundException 

      The command to use to check if a directory exists in Windows PowerShell is Test-Path -Path PATH -PathType Container.

      Fails to launch agent

      [01/24/25 13:52:27] [SSH] Starting agent process: cd "D:\J" && java  -jar remoting.jar -workDir D:\J -jar-cache D:\J/remoting/jarCache
      At line:1 char:11
      + cd "D:\J" && java  -jar remoting.jar -workDir D:\J -jar-cache D:\J/re ...
      +           ~~
      The token '&&' is not a valid statement separator in this version.
          + CategoryInfo          : ParserError: (:) [], ParentContainsErrorRecordException
          + FullyQualifiedErrorId : InvalidEndOfLine 

      Windows PowerShell doesn't support && as command separator. To do this in Windows PowerShell, Set-Location PATH; if ($?) { CMD }{}.

      pwsh Problems

      Escape sequences show in the log:

      [01/24/25 13:55:08] [SSH] The remote user's environment is:
      Set-Variable: Cannot process command because of one or more missing mandatory parameters: Name. 

      The set command doesn't exist

       Set-Variable: Cannot process command because of one or more missing mandatory parameters: Name.
      

      In PowerShell, set is an alias to PowerShell's Set-Variable command. The command to use in PowerShell to view environment variables is Get-ChildItem -Path 'env:'.

      The test command doesn't exist

       test: The term 'test' is not recognized as a name of a cmdlet, function, script file, or executable program.

      The command to use to check if a directory exists in PowerShell is Test-Path -Path PATH -PathType Container.

          [JENKINS-75190] Improve support for connecting to Windows agents using SSH

          Mark Waite added a comment -

          Thanks for reporting the issue and for offering to help with improvements. The ssh build agents plugin source code repository is on GitHub.

          Mark Waite added a comment - Thanks for reporting the issue and for offering to help with improvements. The ssh build agents plugin source code repository is on GitHub .

          Mark Waite added a comment - - edited

          splatteredbits I'm using SSH connections to all but one of my Windows agents using the OpenSSH server provided by Microsoft as part of Windows 10 and Windows 11. I installed and enabled the OpenSSH server from Microsoft, created an account for the agent, generated a private key for that account, added the matching public key to the authorized_keys file of the user, added a Jenkins credential for that private key, then configured an agent to connect to that computer with ssh using that private key.

          I don't see any of the failures that you're mentioning.

          Maybe I've misunderstood and you're wanting the use the ssh-agent functionality to provide a private key to other processes that are running on Windows. If that's the case, then my use of Windows agents running SSH is not what you need. If ssh-agent providing a private key to other processes is what you need, then I assigned this to the wrong plugin. It should be ssh-agent-plugin, just as you assigned it when you created the ticket.

          The ssh-agent-plugin repository is on GitHub.

          Mark Waite added a comment - - edited splatteredbits I'm using SSH connections to all but one of my Windows agents using the OpenSSH server provided by Microsoft as part of Windows 10 and Windows 11. I installed and enabled the OpenSSH server from Microsoft, created an account for the agent, generated a private key for that account, added the matching public key to the authorized_keys file of the user, added a Jenkins credential for that private key, then configured an agent to connect to that computer with ssh using that private key. I don't see any of the failures that you're mentioning. Maybe I've misunderstood and you're wanting the use the ssh-agent functionality to provide a private key to other processes that are running on Windows. If that's the case, then my use of Windows agents running SSH is not what you need. If ssh-agent providing a private key to other processes is what you need, then I assigned this to the wrong plugin. It should be ssh-agent-plugin , just as you assigned it when you created the ticket. The ssh-agent-plugin repository is on GitHub .

          Markus Winter added a comment -

          One thing I noticed in all 3 logs is 

          [01/24/25 13:53:48] [SSH] Starting sftp client. 
          ERROR: [01/24/25 13:53:48] [SSH] SFTP failed. Copying via SCP.

          Only due to that the fallback with "test" and "mkdir -p" is executed. So splatteredbits I would first check why sftp doesn't work.

          That leaves the thing with changing the directory which could be easily overcome by using a prelaunch command  "cd /d d:\j &&"

          Nevertheless I think that having at least detection if it is windows or not would be good to have those fallback commands work and allow to change the drive without a prelaunch command. Not sure if it is really required to support cmd, pwsh and powershell.

          Markus Winter added a comment - One thing I noticed in all 3 logs is  [01/24/25 13:53:48] [SSH] Starting sftp client. ERROR: [01/24/25 13:53:48] [SSH] SFTP failed. Copying via SCP. Only due to that the fallback with "test" and "mkdir -p" is executed. So splatteredbits I would first check why sftp doesn't work. That leaves the thing with changing the directory which could be easily overcome by using a prelaunch command  "cd /d d:\j &&" Nevertheless I think that having at least detection if it is windows or not would be good to have those fallback commands work and allow to change the drive without a prelaunch command. Not sure if it is really required to support cmd, pwsh and powershell.

            Unassigned Unassigned
            splatteredbits Aaron Jensen
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: