Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-22119

Fail to update git repository with stored credential

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Critical Critical
    • git-client-plugin
    • None
    • Jenkins 1.553
      Mac slave

      After upgrade to git plugin 2.0.4, git client plugin 1.6.4
      All the jobs with https repository url and using stored credential failed, refer to following log.

      Downgrade to git:2.0.3, git-client 1.6.3, the issue has gone.

      ====================================================
      00:00:00.002 Started by user anonymous
      00:00:00.003 [EnvInject] - Loading node environment variables.
      00:00:00.004 Building on master in workspace xxx
      00:00:00.108 Fetching changes from the remote Git repository
      00:00:00.141 Fetching upstream changes from https://xxx.git
      00:00:01.667 FATAL: Failed to fetch from https://xxx.git
      00:00:01.668 hudson.plugins.git.GitException: Failed to fetch from https://xxxx.git
      00:00:01.669 at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:621)
      00:00:01.669 at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:853)
      00:00:01.669 at hudson.plugins.git.GitSCM.checkout(GitSCM.java:878)
      00:00:01.669 at hudson.model.AbstractProject.checkout(AbstractProject.java:1414)
      00:00:01.669 at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:671)
      00:00:01.669 at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88)
      00:00:01.669 at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:580)
      00:00:01.669 at hudson.model.Run.execute(Run.java:1676)
      00:00:01.669 at hudson.matrix.MatrixBuild.run(MatrixBuild.java:304)
      00:00:01.669 at hudson.model.ResourceController.execute(ResourceController.java:88)
      00:00:01.669 at hudson.model.Executor.run(Executor.java:231)
      00:00:01.669 at hudson.model.OneOffExecutor.run(OneOffExecutor.java:43)
      00:00:01.669 Caused by: hudson.plugins.git.GitException: Failed to connect to https://xxxx.git using credentials xxxx (status = 401)
      00:00:01.669 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.checkCredentials(CliGitAPIImpl.java:1725)
      00:00:01.669 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1030)
      00:00:01.669 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$200(CliGitAPIImpl.java:88)
      00:00:01.669 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:219)
      00:00:01.669 at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:619)
      00:00:01.669 ... 11 more

          [JENKINS-22119] Fail to update git repository with stored credential

          Mark Waite added a comment -

          I believe the 401 error code in the stack trace indicates that the request was not authorized by the web server. The git-client-plugin 1.6.4 switched from an obsolete version of an http library to a newer supported version of that http library. It also added support the NTLM credentials and for Team Foundation Server git repositories.

          If you'd be willing to perform the test, I'd appreciate having you attempt to upgrade just the git-client-plugin, without upgrading git-plugin. If it fails then that will point to changes in git-client-plugin as the most likely cause. If it passes, then it will need to be investigated in both.

          Does the https server use a self-signed certificate?

          Mark Waite added a comment - I believe the 401 error code in the stack trace indicates that the request was not authorized by the web server. The git-client-plugin 1.6.4 switched from an obsolete version of an http library to a newer supported version of that http library. It also added support the NTLM credentials and for Team Foundation Server git repositories. If you'd be willing to perform the test, I'd appreciate having you attempt to upgrade just the git-client-plugin, without upgrading git-plugin. If it fails then that will point to changes in git-client-plugin as the most likely cause. If it passes, then it will need to be investigated in both. Does the https server use a self-signed certificate?

          Su Shi added a comment -

          yes, with only git-client upgraded to 1.6.4, git fetch failed.

          The server is not using self-signed cert.

          Su Shi added a comment - yes, with only git-client upgraded to 1.6.4, git fetch failed. The server is not using self-signed cert.

          Su Shi added a comment -

          yes, with only git-client upgraded to 1.6.4, git fetch failed.

          The server is not using self-signed cert.

          Su Shi added a comment - yes, with only git-client upgraded to 1.6.4, git fetch failed. The server is not using self-signed cert.

          Mark Waite added a comment -

          Thanks. That indicates the problem is likely in the git-client-plugin changes from 1.6.3 to 1.6.4. We'll need you to do deeper investigation to try to narrow the failure modes. Some of the questions which may help narrow the failure modes include:

          • Does the problem also appear on a copy of the failing job?
          • Does the problem also appear on a newly created job using the same git URL and credentials?
          • Does the problem also appear with other https git repositories in your environment?
          • Does the problem also appear for you with https git repositories outside your environment? (like github or bitbucket)
          • Is there a public URL which can be used to see the same problem?

          Mark Waite added a comment - Thanks. That indicates the problem is likely in the git-client-plugin changes from 1.6.3 to 1.6.4. We'll need you to do deeper investigation to try to narrow the failure modes. Some of the questions which may help narrow the failure modes include: Does the problem also appear on a copy of the failing job? Does the problem also appear on a newly created job using the same git URL and credentials? Does the problem also appear with other https git repositories in your environment? Does the problem also appear for you with https git repositories outside your environment? (like github or bitbucket) Is there a public URL which can be used to see the same problem?

          Mikkel Larsen added a comment -

          I can add a bit of information.

          Rolling back to 1.6.3 solves the problem.

          I get the error pulling from bitbucket, and it fails for every repo I pull from bitbucket.

          Mikkel Larsen added a comment - I can add a bit of information. Rolling back to 1.6.3 solves the problem. I get the error pulling from bitbucket, and it fails for every repo I pull from bitbucket.

          Mark Waite added a comment - - edited

          Thanks for the hint to use bitbucket.

          I can see the failure if I configure a credential with a username and password for a private bitbucket git repository. If the bitbucket repository is public (which I had been using before), then the clone succeeds. More exploration will be required.

          I don't see the same failure if I configure a credential with a username and password for a private github repository. Public github repositories also do not show any issue.

          Provider Private Repo Public Repo
          github 1.6.3 (ok) 1.6.3 (ok)
          bitbucket 1.6.3 (ok) 1.6.3 (ok)
          github 1.6.4 (ok) 1.6.4 (ok)
          bitbucket 1.6.4 (bad) 1.6.4 (ok)

          Mark Waite added a comment - - edited Thanks for the hint to use bitbucket. I can see the failure if I configure a credential with a username and password for a private bitbucket git repository. If the bitbucket repository is public (which I had been using before), then the clone succeeds. More exploration will be required. I don't see the same failure if I configure a credential with a username and password for a private github repository. Public github repositories also do not show any issue. Provider Private Repo Public Repo github 1.6.3 (ok) 1.6.3 (ok) bitbucket 1.6.3 (ok) 1.6.3 (ok) github 1.6.4 (ok) 1.6.4 (ok) bitbucket 1.6.4 (bad) 1.6.4 (ok)

          Darren Warner added a comment - - edited

          Posting a link to my earlier (wire) analysis on this (the bug is not directly related, but for me is dependent on getting this issue fixed): JENKINS-20939 12/Mar/14 3:44 AM

          It appears that the authentication handshaking has changed since the earlier HTTP library

          Darren Warner added a comment - - edited Posting a link to my earlier (wire) analysis on this (the bug is not directly related, but for me is dependent on getting this issue fixed): JENKINS-20939 12/Mar/14 3:44 AM It appears that the authentication handshaking has changed since the earlier HTTP library

          Mikkel Larsen added a comment - - edited

          I finally had some time to dig into this.

          What caused the problem for me was that when you copy the https address from a private repository on bitbucket it is displayed as:

          https://username@bitbucket.org/username/repo.git
          

          and for a public repo it is:

          https://bitbucket.org/username/repo.git
          

          Removing the `username@` from the url in the jenkins setup, fixes the problem for private repos.

          A fix that would allow either format would be to strip the username@ from the url in the `checkCredentials()`: https://github.com/jenkinsci/git-client-plugin/blob/master/src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java#L1854

          e.g something like this:

          diff --git a/src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java b/src/main/java/org/jenkins
          index 9fd48ab..feb95d2 100644
          --- a/src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java
          +++ b/src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java
          @@ -1843,6 +1843,7 @@ public class CliGitAPIImpl extends LegacyCompatibleGitAPIImpl {
                */
               private void checkCredentials(URIish u, StandardCredentials cred) {
                   String url = u.toPrivateString();
          +        url = url.replaceAll("[\\w]+@", "");
                   final HttpClientBuilder clientBuilder = HttpClients.custom();
                   final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
                   clientBuilder.setDefaultCredentialsProvider(credentialsProvider);
          

          Mikkel Larsen added a comment - - edited I finally had some time to dig into this. What caused the problem for me was that when you copy the https address from a private repository on bitbucket it is displayed as: https: //username@bitbucket.org/username/repo.git and for a public repo it is: https: //bitbucket.org/username/repo.git Removing the `username@` from the url in the jenkins setup, fixes the problem for private repos. A fix that would allow either format would be to strip the username@ from the url in the `checkCredentials()`: https://github.com/jenkinsci/git-client-plugin/blob/master/src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java#L1854 e.g something like this: diff --git a/src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java b/src/main/java/org/jenkins index 9fd48ab..feb95d2 100644 --- a/src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java +++ b/src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java @@ -1843,6 +1843,7 @@ public class CliGitAPIImpl extends LegacyCompatibleGitAPIImpl { */ private void checkCredentials(URIish u, StandardCredentials cred) { String url = u.toPrivateString(); + url = url.replaceAll( "[\\w]+@" , ""); final HttpClientBuilder clientBuilder = HttpClients.custom(); final CredentialsProvider credentialsProvider = new BasicCredentialsProvider(); clientBuilder.setDefaultCredentialsProvider(credentialsProvider);

          Mark Waite added a comment -

          I would expect that change to break several other use cases (like the ssh use case I use to push changes back to a GitHub repository). I think users will need to use the work around for the problem case you described (butbucket https URL's behave better without the username embedded in the URL).

          Mark Waite added a comment - I would expect that change to break several other use cases (like the ssh use case I use to push changes back to a GitHub repository). I think users will need to use the work around for the problem case you described (butbucket https URL's behave better without the username embedded in the URL).

          Mikkel Larsen added a comment -

          checkCredentials() seems to only deal with HTTP so I can't see how that would affect ssh?

          Mikkel Larsen added a comment - checkCredentials() seems to only deal with HTTP so I can't see how that would affect ssh?

          Michka Popoff added a comment -

          I just found this bug report as my credentials stopped working with my private bitbucket repo. Removing the @username from the url fixed it.

          This is quite confusing as this was working before and the behavior of the credentials changed between 1.6.3 and 1.6.4
          If the @username can not be stripped, you could perhaps add a hint in the error message telling the user to try without the @username ?

          Michka Popoff added a comment - I just found this bug report as my credentials stopped working with my private bitbucket repo. Removing the @username from the url fixed it. This is quite confusing as this was working before and the behavior of the credentials changed between 1.6.3 and 1.6.4 If the @username can not be stripped, you could perhaps add a hint in the error message telling the user to try without the @username ?

          Javal Nanda added a comment -

          I am facing the same issue not able to connect to repository.
          Git client plugin version is 1.10.2 and git plugin 2.2.5

          I had restored jenkins using thin backup plugin after formatting the mac. Previously everything was working fine but not its not able to connect.

          Javal Nanda added a comment - I am facing the same issue not able to connect to repository. Git client plugin version is 1.10.2 and git plugin 2.2.5 I had restored jenkins using thin backup plugin after formatting the mac. Previously everything was working fine but not its not able to connect.

          Mark Waite added a comment -

          The call to checkCredentials() has been removed in git-client-plugin from 1.13.1 and beyond. Would you be willing to test a pre-release of git-client-plugin 1.13.1 for this case?

          Mark Waite added a comment - The call to checkCredentials() has been removed in git-client-plugin from 1.13.1 and beyond. Would you be willing to test a pre-release of git-client-plugin 1.13.1 for this case?

          Mark Waite added a comment -

          The git-client-plugin 1.14.0 release allows me to fetch and push from a private bitbucket repository using command line git.

          Mark Waite added a comment - The git-client-plugin 1.14.0 release allows me to fetch and push from a private bitbucket repository using command line git.

          Mark Waite added a comment -

          I thought I had a problem with GitHub private repository access through https until I read the GitHub documentation which states that https access to a private repository must use a personal access token. Once I generated and used a personal access token in the credential, it worked as expected.

          Mark Waite added a comment - I thought I had a problem with GitHub private repository access through https until I read the GitHub documentation which states that https access to a private repository must use a personal access token. Once I generated and used a personal access token in the credential, it worked as expected.

            ndeloof Nicolas De Loof
            sushi_k Su Shi
            Votes:
            1 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: