-
Bug
-
Resolution: Fixed
-
Major
-
None
-
Git plugin 2.0, git client plugin 1.4.6, Windows 8, Windows Server 2011, Windows 7
-
Powered by SuggestiMate
A git job configured to use the command line implementation with Git plugin 2.0 and git client plugin 1.4.6 fails to clone on Windows, but successfully clones on Linux.
The problem seems to be that it is trying to configure an environment (setting SSH_PASS=echo) for the launched command, even though Windows does not use the same technique to pass environment variables to a process.
I think there was a different behavior in prior versions of git-client.
I created the job by:
- Configure a global ssh credential
- Create a new job, restrict it to only run on Windows
- Use a git ssh protocol URL (like ssh://wheezy64b/var/cache/git/mwaite/bin.git)
- Select the correct ssh credential from the dropdown list
- Add a build step (I used XShell "echo hello world")
- Save the job
- Run the job
Stack trace on Windows:
Started by user anonymous Building remotely on alan-pc in workspace C:\J\workspace\git-cli-ssh Cloning the remote Git repository Cloning repository ssh://wheezy64b/var/cache/git/mwaite/bin.git git --version git version 1.8.3.msysgit.0 using GIT_SSH to set credentials Jenkins ERROR: Error cloning remote repo 'origin' hudson.plugins.git.GitException: Could not clone ssh://wheezy64b/var/cache/git/mwaite/bin.git at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:310) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:151) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:144) at hudson.remoting.UserRequest.perform(UserRequest.java:118) at hudson.remoting.UserRequest.perform(UserRequest.java:48) at hudson.remoting.Request$2.run(Request.java:326) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at hudson.remoting.Engine$1$1.run(Engine.java:63) at java.lang.Thread.run(Unknown Source) Caused by: hudson.plugins.git.GitException: Command "clone --progress -o origin ssh://wheezy64b/var/cache/git/mwaite/bin.git C:\J\workspace\git-cli-ssh" returned status code 128: stdout: Cloning into 'C:\J\workspace\git-cli-ssh'... stderr: error: cannot spawn C:\Users\Alan\AppData\Local\Temp\ssh3783977685963347919.exe: No such file or directory fatal: unable to fork at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:981) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:920) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$400(CliGitAPIImpl.java:64) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:308) ... 11 more ERROR: null Finished: FAILURE
[JENKINS-20356] Git CLI cannot clone on Windows using GIT_SSH to set credentials when running as a service
Code changed in jenkins
User: Nicolas De Loof
Path:
src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java
http://jenkins-ci.org/commit/git-client-plugin/ba88496975ee2ca04107cfcc0e5b476c53417d66
Log:
JENKINS-20356 windows-compliant GIT_SSH
Code changed in jenkins
User: Nicolas De Loof
Path:
src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java
http://jenkins-ci.org/commit/git-client-plugin/6a44f4acd99d1508945c029d4d92ab20762e4a02
Log:
JENKINS-20356 windows-compliant GIT_SSH
Is there a work around for this issue, or is a code release required in order for this to work. I am getting the same error when trying to connect to a repository in a job configuration.
The only alternatives I've discovered have required that I switch the URL to something other than an ssh URL.
Thanks, unfortunately I have not been very successful with gitlab and http. Commits have usually exceeded size limitations. Did you mention that this was a recent change and prior releases did not behave like this. Is there a version that I can roll back to that would bypass this until this fix is released. I only want to consider forking the branch as a final option.
I believe Git plugin 1.5 worked just fine in this case, as did earlier versions of the plugin, though none of those earlier versions supported the Jenkins credentials store. You'll need to manage the credentials yourself in the job definition, without the convenience of the Jenkins credentials system.
Just to let you know that the build (1.4.7-SNAPSHOT) of the 1.4.x branch fixes this issue for us.
Thanks
Is this fixed in Git Plugin 2.0 yet? I see it is marked as resolved but I am experiencing the exact same problem with a fresh install of the latest Git Plugin (v2.0) and Git Client Plugin (v1.4.6).
After creating some global credentials and using a HTTP URL instead of SSH I got it also working as Mark Waite suggested.
Somehow I managed to get it working using SSH with Jenkins 1.438 (on a Windows Server 2012 in Azure) and Git Plugin 2.0 and GitLab 6.3:
I set the repo url to the SSH url and set it to use no credentials.
Using Git bash I removed the passphrase from the key being used (named id_rsa.pub).
I also set a username and email for the global Git config.
Using GitLab it had to disable "force SSL" (but not disabling this did break my OS X git clone using SSH also).
I hope I did not forget anything...
@carolynvs the fix is not in Git plugin 2.0 yet. The fix is in the source code of the git-client-plugin but a version of the client plugin has not been released which includes the fix. Once that fix is released and verified, then the bug report will be "closed". It is expected that it will be fixed in git client plugin 1.4.7.
Git client plugin 1.5.0 has released. I plan to confirm this bug is fixed in that version.
Upgraded Jenkins GIT client plugin to 1.6.0.
Running Jenkins as Windows service on Windows Server 2012
Started getting the following output:
Building in workspace D:\Jenkins\jobs\Project\workspace Fetching changes from the remote Git repository Fetching upstream changes from ssh://jenkins@gerrit:29418/Project.git using GIT_SSH to set credentials -------------- Killed it after 6 minutes. Then the following output: FATAL: Failed to fetch from ssh://jenkins@gerrit:29418/Project.git hudson.plugins.git.GitException: Failed to fetch from ssh://jenkins@gerrit:29418/Project.git at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:612) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:836) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:861) at hudson.model.AbstractProject.checkout(AbstractProject.java:1412) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:652) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:557) at hudson.model.Run.execute(Run.java:1679) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:230) Caused by: hudson.plugins.git.GitException: Error performing git command at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1108) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:985) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$200(CliGitAPIImpl.java:69) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:218) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.fetch(CliGitAPIImpl.java:224) at hudson.plugins.git.GitAPI.fetch(GitAPI.java:229) at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:610) ... 10 more Caused by: java.lang.InterruptedException at java.lang.ProcessImpl.waitFor(Unknown Source) at hudson.Proc$LocalProc.join(Proc.java:319) at hudson.Proc.joinWithTimeout(Proc.java:168) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1095) ... 16 more
Last build before upgrade had the following output related to Git:
Building in workspace D:\Jenkins\jobs\Project\workspace@2 Fetching changes from the remote Git repository Fetching upstream changes from ssh://jenkins@gerrit:29418/Project.git Checking out Revision bbb4481de2d3f8eb16d5dbf9f5a992c24eb4f1ed (master)
I was able to confirm Hallvard's report that when running Jenkins as a service, it is unable to checkout if the Git URL uses ssh:// (like ssh://wheezy64/var/lib/git/mwaite/bin.git ).
I am still able to checkout if the Git URL uses ssh protocol without the ssh:(like git@github.com:MarkEWaite/jenkins-demo.git ).
I am still able to checkout if Jenkins is not running as a service, even when it uses an ssh:// URL.
It requires both that it is running as a service and that it is using ssh: protocol.
I'm seeing the same issue where the clone fails due to permissions even when the proper SSH key is set for the project. The same project buliding on os x or linux works fine. I think GIT_SSH can't set the ssh key properly before trying to clone on the windows box:
ERROR: Error cloning remote repo 'origin' hudson.plugins.git.GitException: Command "C:\Program Files\Git\cmd\git -c core.askpass=true fetch --tags --progress git@github.com:<github_repo> +refs/heads/*:refs/remotes/origin/*" returned status code 128: stdout: stderr: Permission denied (publickey). fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists.
Versions:
Git version: 2.5.3
Jenkins 1.6.129
Git Plugin 2.4.0
Windows 10 (build-agent – agent installed as windows service)
I'll follow up if I find a workaround.
*UPDATE* - I had to work around this by just deleting the SSH Credential in Jenkins and just setting a public/private key on the windows slave itself (in ~/.ssh) which will be used for all builds. Not ideal as using different SSH keys for building different projects in different git repos is ideal. Perhaps an environment variable isn't being set before trying to run git? Unsure.
I may be experiencing this in what I reported in https://issues.jenkins-ci.org/browse/JENKINS-20941?focusedCommentId=247772&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-247772
There, I have a private github repo, with SSH credentials set, specified as git@github.com:sensics/abcde.git , which I think cloned fine on its own. It has submodules specified, in the same format, for which the same credential should work, in the repo, and the additional submodule behaviors added to the config, but the "access rights" error as seen above occurs for the submodules.
rpavlik: Hey Ryan, so I was having the same issue you were having regarding https://issues.jenkins-ci.org/browse/JENKINS-20941.
So it turns out that Jenkins uses the systemprofile as its default user when it runs. So in order to get this to work I needed to copy my .ssh into the home directory of the systemprofile which for me resides at C:\Windows\system32\config\systemprofile.
But for your specific machine you can run the echo %USERPROFILE$ cmd in a Jenkins job to confirm what that directory is.
Still seeing this myself, using `git@github.com:...` style repo name. I have a multi-platform matrix job that works just fine on linuxes, but fails to checkout the code from windows. I copied out the temp files that get created by the git client plugin to do the GIT_SSH and SSH_ASKPASS magic, and ran it on a command line as a logged in user to get jenkins out of the way, and it appears to totally ignore SSH_ASKPASS. I got prompted every time... This is on git-for-windows 2.9.0. Maybe I should try an older version where this used to work?
And unfortunately, putting the private key on the windows box is not sufficient to override this issue for windows – since the credentials in the jenkins job set the GIT_SSH stuff and overrides whatever it already has locally. So, we'd have to copy the private key to every build node then. This would not be good...
matthauck you might also try to evaluate the git plugin beta and the git client beta that are available from the experimental update center.
This PR solves the issue I am having with windows and GIT_SSH: https://github.com/jenkinsci/git-client-plugin/pull/207
Code changed in jenkins
User: Matt Hauck
Path:
src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java
http://jenkins-ci.org/commit/git-client-plugin/8297db9579996906c8ddfabc470e322951771b84
Log:
JENKINS-20356 Fix SSH_ASKPASS on windows
The SSH_ASKPASS script was being setup improperly on windows since
it was echoing quotes as well as the actual echo command itself
since `@echo off` was not being specified.
Code changed in jenkins
User: Mark Waite
Path:
src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java
http://jenkins-ci.org/commit/git-client-plugin/9c12a007c5a029f92a132b54bf04b5ae31462f2c
Log:
Merge pull request #207 from matthauck/windows-askpass
JENKINS-20356 Fix SSH_ASKPASS on windows
Compare: https://github.com/jenkinsci/git-client-plugin/compare/f52e3e8f2406...9c12a007c5a0
I haven't yet been able to verify that the change in git-client-plugin 2.3.0 is enough to resolve it, but yes, if you've confirmed with git-client-plugin 2.3.0 that it now works as expected on Windows when running as a service, then you could mark it resolved.
Assumed resolved after two years with no further comments. The ssh-slaves plugin now includes instructions to allow recent Windows versions to use the Windows OpenSSH service to run agents.
For those stumbling on this ticket searching for a similar error saying "permission denied", this may result from (domain) administrators installing Bit9 Parity CarbonBlack to white-list the commands allowed on the machine.
ilatypov Could you be more specific? How to know if it's the Bit9 Parity CarbonBlack causes the problem? It would be much helpful if you could share the link about "permission denied", thanks.
One of the machines in my domain is failing because of this reason, the other machine works fine.
The proof was found in Event Viewer / Windows Logs / Application in a message from Source "Cb Protection Agent Notifier".
Notification displayed for target "d:\jenkins\workspace\DIR\PROJ@tmp\jenkins-gitclient-ssh196668178943043519.bat" and process "c:\program files\git\mingw64\bin\git.exe". Cb Protection blocked an attempt by git.exe to run jenkins-gitclient-ssh196668178943043519.bat because the file is not approved. If you require access to this file, please contact your system administrator or submit an approval request. Note that approval requests are processed based on priority and arrival time. Please be patient while your request is reviewed and processed. Scroll down for diagnostic data. Source[c:\program files\git\mingw64\bin\git.exe] ProcessHash[017b2f5aa11781cd293e1c412472ed3d92d08affd945fa63bb3a633b1a98785c] ProcessPublisher[Johannes Schindelin (Valid[Yes] Trusted[Yes])] Cmd[git.exe fetch --tags --force --progress -- ssh://git@COMPANY.TLD:PORT/GROUP/PROJ.git +refs/heads/*:refs/re] ProcessFlags[WrittenFiles:HaveABInfo] KernelProcessFlags[LocalSystem:64Bit:DepEnabled:LocalAdmin] Tags[\device\harddiskvolume1\program files\git\mingw64\bin\git.exe] Target[d:\jenkins\workspace\DIR\PROJ@tmp\jenkins-gitclient-ssh196668178943043519.bat] Notifier[Block] TargetHash[3b29d2bc77bcadb27fc146d767f23d9c46fb5ab7836daa4d0e60134f1e34996b] TargetPublisher[No Publisher (Valid[No] Trusted[Ineligible:No Cert])] Media[Fixed] Device[Unapproved:0x00000000] DeviceFlags[0x00000000] State[Unapproved] Flags[0x00000802] Object[File] Rule[File and Path Execute: Unapproved Executables] List[17] Group[100] Id[27] Server[CBPServer.COMPANY.COM:41002] Policy[COMPANY High Enforcement] Id[41] Version[0x00000000] CLVersion[211507] Enforcement[20:20:20] User[NT AUTHORITY\SYSTEM] Pid[12616] Tid[12936] Computer[XXXXXX] Domain[DDDDDDDD] Agent[8.1.6.212] OS[Microsoft Windows Server 2008 R2 x64 Server Enterprise Service Pack 1 (6.1.7601)] DateTime[3/24/2020 10:03:49 PM]
As a work-around I could replace the default option of using the "git" command with using "JGit" in Global Tool configuration, but because CarbonBlack disabled any other invokation of external commands, I resorted to asking the admins to correct the CarbonBlack limit. I think they added a permission one level above the particular random path to the auto-generated batch files, but I don't know their exact solution. It worked.
Code changed in jenkins
User: Nicolas De Loof
Path:
src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java
http://jenkins-ci.org/commit/git-client-plugin/8035439cc739df3b7814fdb8154b82309974e1a5
Log:
JENKINS-20356windows-compliant GIT_SSHwork in progress, failing now :
Fetching upstream changes from git@github.com:cloudbees/test.git
using GIT_SSH to set credentials
FATAL: Failed to fetch from git@github.com:cloudbees/test.git
hudson.plugins.git.GitException: Failed to fetch from git@github.com:cloudbees/test.git
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:617)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:841)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:866)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1256)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:589)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:494)
at hudson.model.Run.execute(Run.java:1502)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:236)
Caused by: hudson.plugins.git.GitException: Command "fetch -t git@github.com:cloudbees/test.git +refs/heads/:refs/remotes/origin/" returned status code 128:
stdout: Process leaked file descriptors. See http://wiki.jenkins-ci.org/display/JENKINS/Spawning+processes+from+build for more information
stderr: fatal: protocol error: bad line length character:
C:
select: Bad file number