Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-20941

Stored git credentials not used when submodule is updated

      1. Configure a git project that uses a remote and submodule URL of the form https://
      2. Ensure credentials added to Jenkins
      3. Add advanced submodule behaviors with no options selected
      4. First build (where repo is cloned) pulls the submodule correctly
      5. Subsequent builds fail authorization on the submodule part (possibly submodule update if this is being used)

          [JENKINS-20941] Stored git credentials not used when submodule is updated

          Andre Kosiak added a comment -

          markewaite thank you so much! i'm happy!

          Andre Kosiak added a comment - markewaite thank you so much! i'm happy!

          Cameron Gocke added a comment -

          I'm testing with the 3.0.0 beta2 plugin and encountering an error. In particular this is the error reported:
          hudson.plugins.git.GitException: Command "git submodule update --init --recursive php-libraries" returned status code 1:
          stdout:
          stderr: error: pathspec 'php-libraries' did not match any file(s) known to git.

          Now, if I run the same command as stated here from my local git I get the same error, but I can run git submodule init and git submodule update without any issues.

          My git submodule status shows the submodule in question (though with its path, not its submodule name:
          git submodule status
          4f27645ae23a3955d72043afffb75a34449bbbaf var/www/members/libraries (heads/release/v1.0)

          My .gitmodules includes the following, which is correct:
          [submodule "php-libraries"]
          path = var/www/members/libraries
          url = http://blades:7990/scm/phpapi/php-libraries.git
          branch = release/v1.0

          And my .git/config file contains this reference, which seems correct:
          [submodule "php-libraries"]
          url = http://blades:7990/scm/phpapi/php-libraries.git

          Running this from my local git client; however works:
          git submodule update --init --recursive var/www/members/libraries

          So, while not an expert, it seems that the git submodule command is expecting the path, but instead what is being passed to it is the name of the submodule. I'm not sure if the name should work as well, but at least in my case it is not.

          If I arrived at an incorrect conclusion please fill me in on where I went wrong.

          Cameron Gocke added a comment - I'm testing with the 3.0.0 beta2 plugin and encountering an error. In particular this is the error reported: hudson.plugins.git.GitException: Command "git submodule update --init --recursive php-libraries" returned status code 1: stdout: stderr: error: pathspec 'php-libraries' did not match any file(s) known to git. Now, if I run the same command as stated here from my local git I get the same error, but I can run git submodule init and git submodule update without any issues. My git submodule status shows the submodule in question (though with its path, not its submodule name: git submodule status 4f27645ae23a3955d72043afffb75a34449bbbaf var/www/members/libraries (heads/release/v1.0) My .gitmodules includes the following, which is correct: [submodule "php-libraries"] path = var/www/members/libraries url = http://blades:7990/scm/phpapi/php-libraries.git branch = release/v1.0 And my .git/config file contains this reference, which seems correct: [submodule "php-libraries"] url = http://blades:7990/scm/phpapi/php-libraries.git Running this from my local git client; however works: git submodule update --init --recursive var/www/members/libraries So, while not an expert, it seems that the git submodule command is expecting the path, but instead what is being passed to it is the name of the submodule. I'm not sure if the name should work as well, but at least in my case it is not. If I arrived at an incorrect conclusion please fill me in on where I went wrong.

          Mark Waite added a comment - - edited

          camerongo I think you're seeing the same bug described in JENKINS-37495. Unfortunately, I don't know a solution for that problem. I've created a test case for it in my private jenkins-bugs verification repository, but I am unlikely to spend time trying to fix that problem before the release of git plugin 3.0 and git client plugin 2.0.

          Mark Waite added a comment - - edited camerongo I think you're seeing the same bug described in JENKINS-37495 . Unfortunately, I don't know a solution for that problem. I've created a test case for it in my private jenkins-bugs verification repository, but I am unlikely to spend time trying to fix that problem before the release of git plugin 3.0 and git client plugin 2.0.

          Cameron Gocke added a comment -

          Thanks markewaite. At least I know I'm not losing my mind. In the meantime I suppose I can just rename my submodule(s) to work around the issue.

          Cameron Gocke added a comment - Thanks markewaite . At least I know I'm not losing my mind. In the meantime I suppose I can just rename my submodule(s) to work around the issue.

          Mark Waite added a comment - - edited

          Git plugin 3.0.0 and git client plugin 2.0.0 were released 10 Sep 2016 to resolve this issue. Special thanks to Jacob Keller for his patience with a one year beta test, and thanks to those users who performed the beta testing.

          Git plugin 3.0.0 and git client plugin 2.0.0 require a minimum of Jenkins 1.625 and JDK 7.

          Mark Waite added a comment - - edited Git plugin 3.0.0 and git client plugin 2.0.0 were released 10 Sep 2016 to resolve this issue. Special thanks to Jacob Keller for his patience with a one year beta test, and thanks to those users who performed the beta testing. Git plugin 3.0.0 and git client plugin 2.0.0 require a minimum of Jenkins 1.625 and JDK 7.

          rogerdpack added a comment -

          Anybody able/getting this to work? I haven't been (user/password) using what I believe are the latest versions :| https://gist.github.com/rdp/291bda289a94eae96fc32a30e6db9497

          rogerdpack added a comment - Anybody able/getting this to work? I haven't been (user/password) using what I believe are the latest versions :| https://gist.github.com/rdp/291bda289a94eae96fc32a30e6db9497

          Cameron Gocke added a comment -

          rogerdpack yes, it is working fine for me. I installed the 3.0.0 version of the plugin using the Plugin Manager in Jenkins, nothing special. (Note: You have to add the Advanced sub-modules behavior to your SCM config and check the box to "Use credentials from default remote of parent repository".)

          Cameron Gocke added a comment - rogerdpack yes, it is working fine for me. I installed the 3.0.0 version of the plugin using the Plugin Manager in Jenkins, nothing special. (Note: You have to add the Advanced sub-modules behavior to your SCM config and check the box to "Use credentials from default remote of parent repository".)

          Code changed in jenkins
          User: Jacob Keller
          Path:
          src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java
          http://jenkins-ci.org/commit/git-client-plugin/deaf40377a70f9b6995344fbc54a32e29c61098a
          Log:
          git-client: teach submoduleUpdate how to pass credentials

          Teach the submoduleUpdate call to perform submodule URL lookup, and call
          "submodule update" separately for each submodule. Lookup the
          credentials, and call it using launchCommandWithCredentials. This
          enables submodules to correctly get the SSH or HTTP setup necessary such
          that the Jenkins credentials will be passed into each submodule.

          We can't just call "git submodule update" since it may be possible
          (however unlikely!) that each submodule wants to use a separate
          credential. Thus, perform lookup using each URL and run through a
          forloop to actually update each submodule instead of depending on the
          git implementation to do this for us.

          JENKINS-20941 - Credentials and Submodules

          Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jacob Keller Path: src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java http://jenkins-ci.org/commit/git-client-plugin/deaf40377a70f9b6995344fbc54a32e29c61098a Log: git-client: teach submoduleUpdate how to pass credentials Teach the submoduleUpdate call to perform submodule URL lookup, and call "submodule update" separately for each submodule. Lookup the credentials, and call it using launchCommandWithCredentials. This enables submodules to correctly get the SSH or HTTP setup necessary such that the Jenkins credentials will be passed into each submodule. We can't just call "git submodule update" since it may be possible (however unlikely!) that each submodule wants to use a separate credential. Thus, perform lookup using each URL and run through a forloop to actually update each submodule instead of depending on the git implementation to do this for us. JENKINS-20941 - Credentials and Submodules Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>

          Mark Waite added a comment -

          creste3, can you explain why you assigned yourself as owner of this resolved bug?

          Is there some problem that you intend to investigate or resolve?

          Mark Waite added a comment - creste3 , can you explain why you assigned yourself as owner of this resolved bug? Is there some problem that you intend to investigate or resolve?

          John Smith added a comment -

          Sorry, I definitely did not intend to assign myself as the owner of this bug. I suspect that I accidentally pressed one of the JIRA shortcut keys to assign this to myself. I have since disabled keyboard shortcuts because it appears that I am too clumsy to have them enabled.

          There is a problem related to git credentials and submodules that I intend to resolve, but that work should be done in a separate issue because it is related to git LFS.

          John Smith added a comment - Sorry, I definitely did not intend to assign myself as the owner of this bug. I suspect that I accidentally pressed one of the JIRA shortcut keys to assign this to myself. I have since disabled keyboard shortcuts because it appears that I am too clumsy to have them enabled. There is a problem related to git credentials and submodules that I intend to resolve, but that work should be done in a separate issue because it is related to git LFS.

            Unassigned Unassigned
            boinged Steven Shipton
            Votes:
            118 Vote for this issue
            Watchers:
            134 Start watching this issue

              Created:
              Updated:
              Resolved: