Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-63829

Git SSH connection fails in TriLead KexManager

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Minor Minor
    • trilead-api-plugin

      I have a weird problem, that is somhow connected to the TriLead plugin. After we updated to the new LTS version 2.249.1 and updated the plugins. I am no longer able to check out one of my repositories on our Ubuntu 16.04 build agent. If I use an toher build agent everything works OK. It is even stranger, I can even checkout an other repo on the same Agent from the same Bitbucket Server with the same ssh key without problems. Only this one repository wont work. The Bitbucket Server that serves the repoistory is a Bitbucket v6.10.1. If I try to check out the repository, I get the following error:

      using credential my_git_key
      Cloning the remote Git repository
      ERROR: Error cloning remote repo 'origin'
      hudson.plugins.git.GitException: org.eclipse.jgit.api.errors.TransportException: ssh://git@git.intra.home:7999/~myuser/pipelinetest.git: Failed to connect
      	at org.jenkinsci.plugins.gitclient.JGitAPIImpl$4.execute(JGitAPIImpl.java:1490)
      	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161)
      	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:154)
      	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
      	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
      	at hudson.remoting.Request$2.run(Request.java:375)
      	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:73)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      	Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to Ubuntu 16.04 64Bit
      		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1800)
      		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
      		at hudson.remoting.Channel.call(Channel.java:1001)
      		at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
      		at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      		at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      		at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      		at java.lang.reflect.Method.invoke(Method.java:498)
      		at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
      		at com.sun.proxy.$Proxy84.execute(Unknown Source)
      		at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1219)
      		at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1297)
      		at hudson.scm.SCM.checkout(SCM.java:505)
      		at hudson.model.AbstractProject.checkout(AbstractProject.java:1206)
      		at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
      		at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
      		at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
      		at hudson.model.Run.execute(Run.java:1894)
      		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
      		at hudson.model.ResourceController.execute(ResourceController.java:97)
      		at hudson.model.Executor.run(Executor.java:428)
      Caused by: org.eclipse.jgit.api.errors.TransportException: ssh://git@git.intra.home:7999/~myuser/pipelinetest.git: Failed to connect
      	at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:222)
      	at org.jenkinsci.plugins.gitclient.JGitAPIImpl$4.execute(JGitAPIImpl.java:1482)
      	... 10 more
      Caused by: org.eclipse.jgit.errors.TransportException: ssh://git@git.intra.home:7999/~myuser/pipelinetest.git: Failed to connect
      	at org.jenkinsci.plugins.gitclient.trilead.TrileadSessionFactory.getSession(TrileadSessionFactory.java:52)
      	at org.eclipse.jgit.transport.SshTransport.getSession(SshTransport.java:107)
      	at org.eclipse.jgit.transport.TransportGitSsh$SshFetchConnection.<init>(TransportGitSsh.java:254)
      	at org.eclipse.jgit.transport.TransportGitSsh.openFetch(TransportGitSsh.java:144)
      	at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:105)
      	at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:91)
      	at org.eclipse.jgit.transport.Transport.fetch(Transport.java:1260)
      	at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:211)
      	... 11 more
      Caused by: java.io.IOException: There was a problem while connecting to git.intra.home:7999
      	at com.trilead.ssh2.Connection.connect(Connection.java:781)
      	at com.trilead.ssh2.Connection.connect(Connection.java:638)
      	at com.trilead.ssh2.Connection.connect(Connection.java:578)
      	at org.jenkinsci.plugins.gitclient.trilead.TrileadSessionFactory.getSession(TrileadSessionFactory.java:29)
      	... 18 more
      Caused by: java.io.IOException: Key exchange was not finished, connection is closed.
      	at com.trilead.ssh2.transport.KexManager.getOrWaitForConnectionInfo(KexManager.java:95)
      	at com.trilead.ssh2.transport.TransportManager.getConnectionInfo(TransportManager.java:237)
      	at com.trilead.ssh2.Connection.connect(Connection.java:732)
      	... 21 more
      Caused by: java.io.IOException: Fatal error during MAC startup!
      	at com.trilead.ssh2.transport.KexManager.finishKex(KexManager.java:298)
      	at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:571)
      	at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:790)
      	at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:502)
      	... 1 more
      ERROR: Error cloning remote repo 'origin'
      

      If I check out ssh://git@git.intra.home:7999/~myuser/dockertest.git instead of ssh://git@git.intra.home:7999/~myuser/pipelinetest.git, which uses the same ssh key, that works without problems.

          [JENKINS-63829] Git SSH connection fails in TriLead KexManager

          Mark Waite added a comment -

          Thanks for reporting the issues hendi28. The stack trace shows that JGit is being used to fetch that repository.

          Does the working case use command line git rather than JGit? JENKINS-62132 reports that there is a JGit issue with HTTPS based Bitbucket repositories. I've not detected any issues with SSH based Bitbucket repositories

          Is the working case using a repository that is larger than 5 MB while the failing case is a repository smaller than 5 MB? If so, then the git tool chooser could be choosing to use JGit on the small repository and using CLI git on the large repository. You can disable the git tool chooser from "Manage Jenkins" with the git plugin section checkbox "Disable performance enhancements".

          Does JGit succeed when using trilead api plugin 1.0.10?

          Mark Waite added a comment - Thanks for reporting the issues hendi28 . The stack trace shows that JGit is being used to fetch that repository. Does the working case use command line git rather than JGit? JENKINS-62132 reports that there is a JGit issue with HTTPS based Bitbucket repositories. I've not detected any issues with SSH based Bitbucket repositories Is the working case using a repository that is larger than 5 MB while the failing case is a repository smaller than 5 MB? If so, then the git tool chooser could be choosing to use JGit on the small repository and using CLI git on the large repository. You can disable the git tool chooser from "Manage Jenkins" with the git plugin section checkbox "Disable performance enhancements" . Does JGit succeed when using trilead api plugin 1.0.10?

          Hi Mark,

          even if I switch the checkout to use git instead of Jgit, the error stay the same. The stack trace is a bit different, but the originating error stays the same. Here is the stacktrace with git:

          using credential my_git_key
          Fetching changes from the remote Git repository
          ERROR: Error fetching remote repo 'origin'
          hudson.plugins.git.GitException: Failed to fetch from ssh://git@git.intra.home:7999/~myuser/pipelinetest.git
          	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:996)
          	at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1237)
          	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1297)
          	at hudson.scm.SCM.checkout(SCM.java:505)
          	at hudson.model.AbstractProject.checkout(AbstractProject.java:1206)
          	at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
          	at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
          	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
          	at hudson.model.Run.execute(Run.java:1894)
          	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
          	at hudson.model.ResourceController.execute(ResourceController.java:97)
          	at hudson.model.Executor.run(Executor.java:428)
          Caused by: hudson.plugins.git.GitException: org.eclipse.jgit.api.errors.TransportException: ssh://git@git.intra.home:7999/~myuser/pipelinetest.git: Failed to connect
          	at org.jenkinsci.plugins.gitclient.JGitAPIImpl$2.execute(JGitAPIImpl.java:636)
          	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161)
          	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:154)
          	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
          	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
          	at hudson.remoting.Request$2.run(Request.java:375)
          	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:73)
          	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          	at java.lang.Thread.run(Thread.java:748)
          	Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to Ubuntu 16.04 64Bit
          		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1800)
          		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
          		at hudson.remoting.Channel.call(Channel.java:1001)
          		at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
          		at sun.reflect.GeneratedMethodAccessor950.invoke(Unknown Source)
          		at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          		at java.lang.reflect.Method.invoke(Method.java:498)
          		at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
          		at com.sun.proxy.$Proxy77.execute(Unknown Source)
          		at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:994)
          		at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1237)
          		at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1297)
          		at hudson.scm.SCM.checkout(SCM.java:505)
          		at hudson.model.AbstractProject.checkout(AbstractProject.java:1206)
          		at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
          		at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
          		at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
          		at hudson.model.Run.execute(Run.java:1894)
          		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
          		at hudson.model.ResourceController.execute(ResourceController.java:97)
          		at hudson.model.Executor.run(Executor.java:428)
          Caused by: org.eclipse.jgit.api.errors.TransportException: ssh://git@git.intra.home:7999/~myuser/pipelinetest.git: Failed to connect
          	at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:222)
          	at org.jenkinsci.plugins.gitclient.JGitAPIImpl$2.execute(JGitAPIImpl.java:634)
          	... 10 more
          Caused by: org.eclipse.jgit.errors.TransportException: ssh://git@git.intra.home:7999/~myuser/pipelinetest.git: Failed to connect
          	at org.jenkinsci.plugins.gitclient.trilead.TrileadSessionFactory.getSession(TrileadSessionFactory.java:52)
          	at org.eclipse.jgit.transport.SshTransport.getSession(SshTransport.java:107)
          	at org.eclipse.jgit.transport.TransportGitSsh$SshFetchConnection.<init>(TransportGitSsh.java:254)
          	at org.eclipse.jgit.transport.TransportGitSsh.openFetch(TransportGitSsh.java:144)
          	at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:105)
          	at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:91)
          	at org.eclipse.jgit.transport.Transport.fetch(Transport.java:1260)
          	at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:211)
          	... 11 more
          Caused by: java.io.IOException: There was a problem while connecting to git.intra.home:7999
          	at com.trilead.ssh2.Connection.connect(Connection.java:781)
          	at com.trilead.ssh2.Connection.connect(Connection.java:638)
          	at com.trilead.ssh2.Connection.connect(Connection.java:578)
          	at org.jenkinsci.plugins.gitclient.trilead.TrileadSessionFactory.getSession(TrileadSessionFactory.java:29)
          	... 18 more
          Caused by: java.io.IOException: Key exchange was not finished, connection is closed.
          	at com.trilead.ssh2.transport.KexManager.getOrWaitForConnectionInfo(KexManager.java:95)
          	at com.trilead.ssh2.transport.TransportManager.getConnectionInfo(TransportManager.java:237)
          	at com.trilead.ssh2.Connection.connect(Connection.java:732)
          	... 21 more
          Caused by: java.io.IOException: Fatal error during MAC startup!
          	at com.trilead.ssh2.transport.KexManager.finishKex(KexManager.java:298)
          	at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:571)
          	at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:790)
          	at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:502)
          	... 1 more
          ERROR: Error fetching remote repo 'origin'
          
          

          Hendrik Brandenburger added a comment - Hi Mark, even if I switch the checkout to use git instead of Jgit, the error stay the same. The stack trace is a bit different, but the originating error stays the same. Here is the stacktrace with git: using credential my_git_key Fetching changes from the remote Git repository ERROR: Error fetching remote repo 'origin' hudson.plugins.git.GitException: Failed to fetch from ssh://git@git.intra.home:7999/~myuser/pipelinetest.git at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:996) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1237) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1297) at hudson.scm.SCM.checkout(SCM.java:505) at hudson.model.AbstractProject.checkout(AbstractProject.java:1206) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499) at hudson.model.Run.execute(Run.java:1894) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:428) Caused by: hudson.plugins.git.GitException: org.eclipse.jgit.api.errors.TransportException: ssh://git@git.intra.home:7999/~myuser/pipelinetest.git: Failed to connect at org.jenkinsci.plugins.gitclient.JGitAPIImpl$2.execute(JGitAPIImpl.java:636) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:154) at hudson.remoting.UserRequest.perform(UserRequest.java:211) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:375) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:73) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to Ubuntu 16.04 64Bit at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1800) at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357) at hudson.remoting.Channel.call(Channel.java:1001) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146) at sun.reflect.GeneratedMethodAccessor950.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132) at com.sun.proxy.$Proxy77.execute(Unknown Source) at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:994) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1237) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1297) at hudson.scm.SCM.checkout(SCM.java:505) at hudson.model.AbstractProject.checkout(AbstractProject.java:1206) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499) at hudson.model.Run.execute(Run.java:1894) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:428) Caused by: org.eclipse.jgit.api.errors.TransportException: ssh://git@git.intra.home:7999/~myuser/pipelinetest.git: Failed to connect at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:222) at org.jenkinsci.plugins.gitclient.JGitAPIImpl$2.execute(JGitAPIImpl.java:634) ... 10 more Caused by: org.eclipse.jgit.errors.TransportException: ssh://git@git.intra.home:7999/~myuser/pipelinetest.git: Failed to connect at org.jenkinsci.plugins.gitclient.trilead.TrileadSessionFactory.getSession(TrileadSessionFactory.java:52) at org.eclipse.jgit.transport.SshTransport.getSession(SshTransport.java:107) at org.eclipse.jgit.transport.TransportGitSsh$SshFetchConnection.<init>(TransportGitSsh.java:254) at org.eclipse.jgit.transport.TransportGitSsh.openFetch(TransportGitSsh.java:144) at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:105) at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:91) at org.eclipse.jgit.transport.Transport.fetch(Transport.java:1260) at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:211) ... 11 more Caused by: java.io.IOException: There was a problem while connecting to git.intra.home:7999 at com.trilead.ssh2.Connection.connect(Connection.java:781) at com.trilead.ssh2.Connection.connect(Connection.java:638) at com.trilead.ssh2.Connection.connect(Connection.java:578) at org.jenkinsci.plugins.gitclient.trilead.TrileadSessionFactory.getSession(TrileadSessionFactory.java:29) ... 18 more Caused by: java.io.IOException: Key exchange was not finished, connection is closed. at com.trilead.ssh2.transport.KexManager.getOrWaitForConnectionInfo(KexManager.java:95) at com.trilead.ssh2.transport.TransportManager.getConnectionInfo(TransportManager.java:237) at com.trilead.ssh2.Connection.connect(Connection.java:732) ... 21 more Caused by: java.io.IOException: Fatal error during MAC startup! at com.trilead.ssh2.transport.KexManager.finishKex(KexManager.java:298) at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:571) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:790) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:502) ... 1 more ERROR: Error fetching remote repo 'origin'

          If I disable the "performance enhancements", it starts working. I will try to test with trilead 1.0.10 later this evening.

          Hendrik Brandenburger added a comment - If I disable the "performance enhancements", it starts working. I will try to test with trilead 1.0.10 later this evening.

          Mark Waite added a comment -

          Thanks hendi28. I'm reasonably confident that trilead 1.0.10 will resolve the issue. By disabling the performance enhancements, you switched from JGit (selected by the git tool chooser based on repository size) to CLI git (as selected by your job configuration). JGit uses the trilead-api-plugin for its SSH connection. Command line git uses the ssh command on the controller (or agent) for its SSH connection.

          Mark Waite added a comment - Thanks hendi28 . I'm reasonably confident that trilead 1.0.10 will resolve the issue. By disabling the performance enhancements, you switched from JGit (selected by the git tool chooser based on repository size) to CLI git (as selected by your job configuration). JGit uses the trilead-api-plugin for its SSH connection. Command line git uses the ssh command on the controller (or agent) for its SSH connection.

          Rashid Amiri added a comment -

          We had an issue connecting to our Subversion server and downgrading to 1.0.10 helped. 

          Rashid Amiri added a comment - We had an issue connecting to our Subversion server and downgrading to 1.0.10 helped. 

            Unassigned Unassigned
            hendi28 Hendrik Brandenburger
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: