Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-51907

Git fetch with submodules fails when an submodule has different commit --no-recurse-submodules needed

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Minor
    • Resolution: Unresolved
    • Component/s: git-plugin
    • Labels:
      None
    • Environment:
      Jenkins ver. 2.121.1, Git Plugin 3.9.1
    • Similar Issues:

      Description

      It is my opinion is that when the "Disable submodules processing" is set the git fetch should have the `--no-recurse-submodules` option. Here is why:

      We have a multibranch pipeline: which we are using for a git repo that has a number of submodules.

      Due to security reasons, we do not give Jenkins permission to all the submodules via one ssh key. As such we load each submodule separately in the Jenkinsfile with the individual ssh key. This works great. 

      However, when we push a different commit to one of the submodules  Jenkins always fails the first time that it runs that job. With this error: 
       

      /opt/bitnami/git/bin/git fetch --no-tags --progress git@github.com:repo/repo.git +refs/heads/*:refs/remotes/origin/*
      ERROR: Error fetching remote repo 'origin'
      hudson.plugins.git.GitException: Failed to fetch from git@github.com:repo/repo.git
      	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
      	at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
      	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
      	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:113)
      	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:85)
      	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:75)
      	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
      	at hudson.security.ACL.impersonate(ACL.java:290)
      	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      Caused by: hudson.plugins.git.GitException: Command "/opt/bitnami/git/bin/git fetch --no-tags --progress git@github.com:repo/repo.git +refs/heads/*:refs/remotes/origin/*" returned status code 1:
      

      As you can see this is an error with the git command fetch. It would seem that when one of the submodules has a change in the commit the fetch will do a recursive fetch. You can see that this is the default behaviour.. As such due to the lack of permissions that the key has for the submodules it fails. This is easily solved with the addition `--no-recurse-submodules` to the fetch command.

      My opinion is that when the "Disable submodules processing" is set the git fetch should have the `--no-recurse-submodules` option. As such solving the problems described above.

        Attachments

          Activity

          Hide
          jtancer Jon Tancer added a comment -

          +1 We are experiencing the same issue, although we don't use different SSH keys for each submodule.

          We are using a GitHub organization which under the hood uses a token (basic http auth) to authenticate to GitHub when pulling. Our submodules are referenced using SSH, so this causes our pipelines to fail as well the first time.

          Benjamin Cerigo - have you made any progress on finding a workaround for your issue?

          Show
          jtancer Jon Tancer added a comment - +1 We are experiencing the same issue, although we don't use different SSH keys for each submodule. We are using a GitHub organization which under the hood uses a token (basic http auth) to authenticate to GitHub when pulling. Our submodules are referenced using SSH, so this causes our pipelines to fail as well the first time. Benjamin Cerigo - have you made any progress on finding a workaround for your issue?
          Hide
          benjamincerigo Benjamin Cerigo added a comment -

          Hi Jon, Unfortunately, we never found a workaround. We just ended up rerunning by hand on fail.  

          Show
          benjamincerigo Benjamin Cerigo added a comment - Hi Jon, Unfortunately, we never found a workaround. We just ended up rerunning by hand on fail.  
          Hide
          jtancer Jon Tancer added a comment -

          That is unfortunate Benjamin.  For now, I'll try to change the GitHub organization to clone with SSH and tick off the option "Use credentials from default remote of parent repository".

          Show
          jtancer Jon Tancer added a comment - That is unfortunate Benjamin.  For now, I'll try to change the GitHub organization to clone with SSH and tick off the option "Use credentials from default remote of parent repository".
          Hide
          benjamincerigo Benjamin Cerigo added a comment -

          That seems like a good option. Best of luck. 

          Show
          benjamincerigo Benjamin Cerigo added a comment - That seems like a good option. Best of luck. 

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            benjamincerigo Benjamin Cerigo
            Votes:
            4 Vote for this issue
            Watchers:
            5 Start watching this issue

              Dates

              Created:
              Updated: