Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-57936

Parameters not expanded in username or password portion of https git URL

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Minor Minor
    • git-client-plugin
    • None
    • Jenkins 2.164.3 git plugin 3.10.0 git-client 2.7.7

      When a parameterized job attempts to use the value of the parameter as the username or password of an https based git repository URL, the git client plugin does not expand the parameter reference. If a string parameter USERNAME and a password parameter PASSWORD are defined in either a Freestyle or a Pipeline job, references to those variables are not expanded in the username and password portion of the repository URL.

      https://${USERNAME}:${PASSWORD}@github.com/${USERNAME}/secret-repo.git
      

      becomes:

      https://${USERNAME}:${PASSWORD}@github.com/MarkEWaite/secret-repo.git
      

      Notice that the USERNAME parameter is replaced in some parts of the URL, but not in all parts of the URL.


      Dear Jenkins users,

      I'm use Jenkins 2.164.3 and trying to clone a repository with Git Plugin I get the error " authentication failed", because for some reason git plugin add "%24" in front of my password read from password parameter. 

      I read from ticket JENKINS-20533

      It was an old bug of 5 years ago of very old git plugin, but now should be solved!

      Could you help me? Thanks.

          [JENKINS-57936] Parameters not expanded in username or password portion of https git URL

          Mark Waite added a comment - - edited

          unrue please provide a numbered series of steps which will duplicate the problem you are seeing. As an example of the steps that I use to test various authentication schemes with the Jenkins git plugin, I use:

          1. Define a Jenkins username / password credential using a valid username and password for the git account that will provide the repository
          2. Define a Freestyle job that uses the https protocol to clone that repository and users the credential defined in the previous step
          3. Run the Freestyle job to confirm that the https protocol clone works as expected.

          You say that

          for some reason git plugin add "%24" in front of my password read from password parameter.

          How can you see that Jenkins is adding "%24" to the front of your password? Are you cloning without using the git plugin? Does your password start with the character '$' (which might be HTML escaped to '%24')?

          Are you cloning an authenticated repository by embedding the username and password into the URL instead of using a credential? If so, does it work any better if you use a Jenkins credential instead of embedding the username and password in the repository URL?

          Mark Waite added a comment - - edited unrue please provide a numbered series of steps which will duplicate the problem you are seeing. As an example of the steps that I use to test various authentication schemes with the Jenkins git plugin, I use: Define a Jenkins username / password credential using a valid username and password for the git account that will provide the repository Define a Freestyle job that uses the https protocol to clone that repository and users the credential defined in the previous step Run the Freestyle job to confirm that the https protocol clone works as expected. You say that for some reason git plugin add "%24" in front of my password read from password parameter. How can you see that Jenkins is adding "%24" to the front of your password? Are you cloning without using the git plugin? Does your password start with the character '$' (which might be HTML escaped to '%24')? Are you cloning an authenticated repository by embedding the username and password into the URL instead of using a credential? If so, does it work any better if you use a Jenkins credential instead of embedding the username and password in the repository URL?

          Gabriele Fatigati added a comment - - edited

          @Mark White, my steps:

          • Created new freestyle jobs 
          • Set password parameter
          • Pass password parameter to Jenkins Git form

          My password does not start with '$', but it is a job password parameter, so in my opinion Jenkins fail expanding such variable. The same problem appears on Hudson some years ago.

          The "%24" is shown from the build log

          Gabriele Fatigati added a comment - - edited @Mark White, my steps: Created new freestyle jobs  Set password parameter Pass password parameter to Jenkins Git form My password does not start with '$', but it is a job password parameter, so in my opinion Jenkins fail expanding such variable. The same problem appears on Hudson some years ago. The "%24" is shown from the build log

          Mark Waite added a comment - - edited

          Thanks for the steps. I've confirmed that the bug can be duplicated. Steps I took:

          1. Define a parameterized job with two parameters, a String parameter USERNAME with default value MarkEWaite and a Password parameter PASSWORD with default value of a newly created GitHub personal access token
          2. Define the git repository for that job as https://${USERNAME}:${PASSWORD}@github.com/${USERNAME}/jenkins-bugs-private
          3. Run the job

          The job fails and provides the following output:

          No credentials specified
           > git rev-parse --is-inside-work-tree # timeout=10
          Fetching changes from the remote Git repository
           > git config remote.origin.url https://%24{USERNAME}:%24{PASSWORD}@github.com/MarkEWaite/jenkins-bugs-private # timeout=10
          Fetching upstream changes from https://%24{USERNAME}@github.com/MarkEWaite/jenkins-bugs-private
           > git --version # timeout=10
           > git fetch --no-tags --force --progress https://%24{USERNAME}@github.com/MarkEWaite/jenkins-bugs-private +refs/heads/master:refs/remotes/origin/master # timeout=10
          ERROR: Error fetching remote repo 'origin'
          hudson.plugins.git.GitException: Failed to fetch from https://%24{USERNAME}@github.com/MarkEWaite/jenkins-bugs-private
          	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:904)
          	at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1119)
          	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1150)
          	at hudson.scm.SCM.checkout(SCM.java:504)
          	at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
          	at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
          	at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
          	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
          	at hudson.model.Run.execute(Run.java:1818)
          	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
          	at hudson.model.ResourceController.execute(ResourceController.java:97)
          	at hudson.model.Executor.run(Executor.java:429)
          Caused by: hudson.plugins.git.GitException: Command "git fetch --no-tags --force --progress https://%24{USERNAME}@github.com/MarkEWaite/jenkins-bugs-private +refs/heads/master:refs/remotes/origin/master" returned status code 128:
          stdout: 
          stderr: remote: Invalid username or password.
          fatal: Authentication failed for 'https://%24{USERNAME}@github.com/MarkEWaite/jenkins-bugs-private/'
          
          	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2298)
          	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1910)
          	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$400(CliGitAPIImpl.java:81)
          	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:488)
          	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161)
          	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:154)
          	at hudson.remoting.UserRequest.perform(UserRequest.java:212)
          	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
          	at hudson.remoting.Request$2.run(Request.java:369)
          	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
          	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          	at java.lang.Thread.run(Thread.java:748)
          	Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to testing-a-jagent
          		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1743)
          		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
          		at hudson.remoting.Channel.call(Channel.java:957)
          		at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
          		at sun.reflect.GeneratedMethodAccessor535.invoke(Unknown Source)
          		at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          		at java.lang.reflect.Method.invoke(Method.java:498)
          		at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
          		at com.sun.proxy.$Proxy99.execute(Unknown Source)
          		at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:902)
          		at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1119)
          		at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1150)
          		at hudson.scm.SCM.checkout(SCM.java:504)
          		at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
          		at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
          		at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
          		at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
          		at hudson.model.Run.execute(Run.java:1818)
          		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
          		at hudson.model.ResourceController.execute(ResourceController.java:97)
          		at hudson.model.Executor.run(Executor.java:429)
          ERROR: Error fetching remote repo 'origin'
          

          A similar stack trace is reported when JGit is used as the implementation instead of command line git.

          One of the parameters is replaced in the repository URL (the user name after the hostname in the URL) while the other two parameters are not replaced in the repository URL.

          Even embedding the username and password directly into the repository URL does not work for cloning from GitHub. Thus, there are at least two bugs here, first that parameter expansion is not being performed in the optional username/password portion of the repository URL and second that a username and password embedded in a repository URL are not supported.

          Mark Waite added a comment - - edited Thanks for the steps. I've confirmed that the bug can be duplicated. Steps I took: Define a parameterized job with two parameters, a String parameter USERNAME with default value MarkEWaite and a Password parameter PASSWORD with default value of a newly created GitHub personal access token Define the git repository for that job as https://$ {USERNAME}:${PASSWORD}@github.com/${USERNAME}/jenkins-bugs-private Run the job The job fails and provides the following output: No credentials specified > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url https://%24{USERNAME}:%24{PASSWORD}@github.com/MarkEWaite/jenkins-bugs-private # timeout=10 Fetching upstream changes from https://%24{USERNAME}@github.com/MarkEWaite/jenkins-bugs-private > git --version # timeout=10 > git fetch --no-tags --force --progress https://%24{USERNAME}@github.com/MarkEWaite/jenkins-bugs-private +refs/heads/master:refs/remotes/origin/master # timeout=10 ERROR: Error fetching remote repo 'origin' hudson.plugins.git.GitException: Failed to fetch from https://%24{USERNAME}@github.com/MarkEWaite/jenkins-bugs-private at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:904) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1119) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1150) at hudson.scm.SCM.checkout(SCM.java:504) at hudson.model.AbstractProject.checkout(AbstractProject.java:1208) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499) at hudson.model.Run.execute(Run.java:1818) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) Caused by: hudson.plugins.git.GitException: Command "git fetch --no-tags --force --progress https://%24{USERNAME}@github.com/MarkEWaite/jenkins-bugs-private +refs/heads/master:refs/remotes/origin/master" returned status code 128: stdout: stderr: remote: Invalid username or password. fatal: Authentication failed for 'https://%24{USERNAME}@github.com/MarkEWaite/jenkins-bugs-private/' at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2298) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1910) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$400(CliGitAPIImpl.java:81) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:488) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:154) at hudson.remoting.UserRequest.perform(UserRequest.java:212) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:369) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to testing-a-jagent at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1743) at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357) at hudson.remoting.Channel.call(Channel.java:957) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146) at sun.reflect.GeneratedMethodAccessor535.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132) at com.sun.proxy.$Proxy99.execute(Unknown Source) at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:902) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1119) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1150) at hudson.scm.SCM.checkout(SCM.java:504) at hudson.model.AbstractProject.checkout(AbstractProject.java:1208) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499) at hudson.model.Run.execute(Run.java:1818) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) ERROR: Error fetching remote repo 'origin' A similar stack trace is reported when JGit is used as the implementation instead of command line git. One of the parameters is replaced in the repository URL (the user name after the hostname in the URL) while the other two parameters are not replaced in the repository URL. Even embedding the username and password directly into the repository URL does not work for cloning from GitHub. Thus, there are at least two bugs here, first that parameter expansion is not being performed in the optional username/password portion of the repository URL and second that a username and password embedded in a repository URL are not supported.

          Hi,

          I am facing the same issue, I generated a personal access token and using that as a credential parameter(secret text) in Git plugin like https://$TOKEN@github.com but its not able to resolve the variable and if I dont use the variable then it will appear in console output.

          Do we have a possible solution yet?

           

          Thanks,

          Piyush

          piyush sachdeva added a comment - Hi, I am facing the same issue, I generated a personal access token and using that as a credential parameter(secret text) in Git plugin like https://$TOKEN@github.com  but its not able to resolve the variable and if I dont use the variable then it will appear in console output. Do we have a possible solution yet?   Thanks, Piyush

          Mark Waite added a comment - - edited

          piyush_devops there is already a solution for that case. Create a Jenkins username/password credential which contains the username and the personal access token. Use that credential in your operations. The value of the credential is not displayed when a credential is used in a Jenkins job.

          Refer to "Using Credentials" in the Jenkins Handbook for more information.

          Mark Waite added a comment - - edited piyush_devops there is already a solution for that case. Create a Jenkins username/password credential which contains the username and the personal access token. Use that credential in your operations. The value of the credential is not displayed when a credential is used in a Jenkins job. Refer to " Using Credentials " in the Jenkins Handbook for more information.

          Hi Mark,

          Thanks for your reply however, I have tried the approach already but I am still getting the below error

          fatal: could not read Username for 'https://github.xx.com': terminal prompts disabled.

          PS: My Jenkins is running as a Docker container/

          piyush sachdeva added a comment - Hi Mark, Thanks for your reply however, I have tried the approach already but I am still getting the below error fatal: could not read Username for ' https://github.xx.com ': terminal prompts disabled. PS: My Jenkins is running as a Docker container/

          Mark Waite added a comment -

          piyush_devops please don't use a bug report as a place to request help with configuration issues. This bug report describes a failure to expand a parameter in a git URL that is provided to a Freestyle job.

          Your issue is different and should be handled through the user mailing list or the chat system. Many, many users are successfully cloning repositories with GitHub personal access tokens. The most likely problem is a configuration error somewhere in your environment. That's not a bug, it is a configuration error.

          Mark Waite added a comment - piyush_devops please don't use a bug report as a place to request help with configuration issues. This bug report describes a failure to expand a parameter in a git URL that is provided to a Freestyle job. Your issue is different and should be handled through the user mailing list or the chat system. Many, many users are successfully cloning repositories with GitHub personal access tokens. The most likely problem is a configuration error somewhere in your environment. That's not a bug, it is a configuration error.

          markewaite, the reason I am commenting on this thread is because the status of this bug is still unresolved and I am facing the exact same issue.

          piyush sachdeva added a comment - markewaite , the reason I am commenting on this thread is because the status of this bug is still unresolved and I am facing the exact same issue.

          Mark Waite added a comment - - edited

          piyush_devops if you are passing the GitHub personal access token as a parameter to a job, you're making a mistake. Don't do that.

          Job parameters are visible from the "Parameters" link on the left-hand side of the individual job history. In my case, job #7 shows the parameters which were used to run that job. If a personal access token had been entered into a parameter field to run that job, the value of the personal access token would be visible from that page.

          Please use Jenkins credentials. They are designed to store credentials. They work well with Jenkins plugins. Investigate why your credentials are not working. Don't pass sensitive text as a parameter to a Jenkins job.

          Mark Waite added a comment - - edited piyush_devops if you are passing the GitHub personal access token as a parameter to a job, you're making a mistake. Don't do that. Job parameters are visible from the "Parameters" link on the left-hand side of the individual job history. In my case, job #7 shows the parameters which were used to run that job. If a personal access token had been entered into a parameter field to run that job, the value of the personal access token would be visible from that page. Please use Jenkins credentials. They are designed to store credentials. They work well with Jenkins plugins. Investigate why your credentials are not working. Don't pass sensitive text as a parameter to a Jenkins job.

          markewaite, yes that is why I was using Jenkins credential store along with the masked variable but it's not able to resolve the variable. I don't want it to print in Jenkins console output that is why I have posted in this thread(my first comment). I will keep digging into it. 

          Thanks

          piyush sachdeva added a comment - markewaite , yes that is why I was using Jenkins credential store along with the masked variable but it's not able to resolve the variable. I don't want it to print in Jenkins console output that is why I have posted in this thread(my first comment). I will keep digging into it.  Thanks

            unrue Gabriele Fatigati
            unrue Gabriele Fatigati
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: